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EXECUTIVE SUMMARY 

This report describes the conduct and outcomes of an experimental pilot study conducted in 
Spring 2004 to develop and test a model that aimed to enhance career and technical education 
(CTE) instruction with the mathematics that is already embedded in the curricula of six 
occupational areas. 

Math is abundant in the CTE curriculum, but it is largely implicit to both teachers and 
students. The impetus for the study is that many high school students, particularly those in 
enrolled in CTE courses, do not have the math skills necessary for today’s jobs or college 
entrance requirements. This research project was aimed at using an authentic context for teaching 
math skills. Preparation for the study began in the summer of 2003 with the nationwide 
recruitment of teacher-participants. CTE teachers who were interested in participating were 
required to identify teachers of mathematics who was willing to work with them during the 
course of the study. Erom a total of 274 CTE teachers who applied to participate, 114 were 
randomly assigned to and participated in the experimental group, whereas the other 122 served 
as controls. 

In the fall of 2003, the experimental CTE teachers and their math-teacher partners attended a 
professional development workshop for their occupational area. At these workshops, the 
CTE-math teacher teams identified the mathematical concepts in the curricula of the CTE 
teachers and developed lessons to provide explicit instruction in these concepts. The lessons 
were required to incorporate the following elements: 

1. “pull out” the mathematics found in the CTE context 

2. assess students’ math understanding 

3. work through the pulled-out example 

4. identify the underlying math concept, using math vocabulary 

5. work through similar examples and generic examples 

6. check for understanding 

7. have students create other examples, both from CTE and from traditional math 

In the 2nd half of the 2003-2004 school year, the experimental teachers taught the lessons 
developed by the group of teacher teams in the fall workshop for their occupational area. 
Pretesting with one standardized mathematics test (TerraNova) was conducted prior to the first 
lesson, and posttesting was conducted after all had been taught. Three different types of math 
posttests were administered: another form of TerraNova, ACCUPLACER, and WorkKeys®. 
Classes were randomly divided so only one third of each class took each one of these tests. In 
addition, students in each of the six occupational areas took a posttest that assessed their 
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knowledge and skills in that area. These tests were administered to determine whether or not the 
instruction time used for enhancing math was detrimental to the learning of the CTE content. 

Both quantitative and qualitative data were collected and analyzed to assess fidelity of the 
treatment and to gain understanding about the teacher experiences during implementation of the 
math-enhancement model. Teacher surveys, interviews, and focus groups were conducted. Math 
teachers were asked to meet with the CTE teachers after the lessons had been taught and submit 
debriefing reports. Additionally, each teacher was observed once during the semester by a 
member of the research team, and instructional artifacts were collected from each classroom. 

Random assignment was made at the classroom, and not the individual student, level; the unit 
of analysis was the classroom. Despite random assignment, the pretest yielded significant 
differences in the average math scores of the experimental and control classrooms in two of the 
six occupational areas, but not overall. Because of these differences, the pretest was used as a 
covariate in analyzing the posttest scores. These analyses showed a significant difference 
(p < .10) in favor of the experimental group on the ACCUPLACER test (mean effect size = .20). 
Analyses of the six separate sites on the three math posttests found that 14 of the 18 differences 
favored the experimental group. The nonparametric sign test indicates that such a pattern has a 
probability of less than .04. Examining sites separately, two of the six sites had significant 
differences in favor of the experimental group on ACCUPLACER (Site A, effect size = .32; and 
Site C, effect size = .46). Site C also had a significant difference on WorkKeys (effect size .40). 
These improvements in math achievement did not come at the cost of lower scores on the tests of 
occupational skills and knowledge. At the classroom level, there were no significant differences 
between the experimental and control groups on these tests. 

A review of the formative evaluation data assembled as part of the pilot study led to a 
number of changes in preparation for the full-year study (2004-2005 school year). These 
included revisions in the seven-element pedagogic model and in the amount and kind of math 
support provided to the CTE instructors. The revisions in the model emphasized more bridging 
between the CTE and mathematics vocabularies and increased attention to how the embedded 
math is represented in traditional math instruction. Increased math support was provided through 
additional extended professional development workshops, cluster meetings of small groups 
between the workshops, Web sites with resources for each of the occupational areas, and a 
reporting system for monitoring the collaboration between the CTE and math instructors. The 
full-year study was implemented in five of the original six occupational areas. Pretesting was 
conducted in the fall of 2004 and posttesting was conducted in late spring 2005. The report of the 
full-year study will be available at the end of 2005. 
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CHAPTER 1: INTRODUCTION 

The forces of technology, demographics, and global economic competition are combining in 
unprecedented ways to change work and redefine the American workplace. Increasingly, 
employers are demanding higher levels of problem-solving skills from their workers. Unlike jobs 
a half-century ago, most positions today that pay family-supporting wages and offer 
opportunities for advancement demand strong academic and technical skills, technological 
proficiency, and some education beyond high school. A recent report from ACT (2004a) found 
that most students are not being prepared to meet these demands: 

ACT research shows that far too few members of the graduating class of 2004 are 
ready for college-level work in English, math, or science — or for the workplace, 
where the same skills are now being expected of those who do not attend college. 

This deficiency is evident among both males and females and among all racial 
and ethnic groups. And, at present, it does not look as though students already in 
the pipeline are likely to fare much better, (p. 1) 

Every student must be well-prepared to adapt and adjust to the ever-changing economy in 
order to choose a career freely, and that may include jobs not even present among today’s 
options. The one constant message of the past 20 years of education reform is that high schools 
must strengthen the academic performance of all students. 

In 2000, the National Council of Teachers of Mathematics (NCTM) issued a report that 
emphasized math as one of the “new basic skills” for industry. Mathematics is no longer a 
requirement only for prospective scientists and engineers. Instead, some degree of mathematical 
literacy is required of anyone entering a workplace or seeking advancement in a career 
(Mathematical Sciences Education Board, 1995). Research by Levy and Murnane (2004) has 
shown that higher wages are associated with the ability to think mathematically. 

The calls for reform are continuous, as are employers’ complaints about the difficulty of 
hiring young people who have the right skills: 

Seventy-eight percent of respondents believe public schools are failing to prepare 
students for the workplace, which represents little change from the 1991 and 1997 
surveys, despite a decade of various education reform movements. Respondents 
said the biggest deficiency of public schools is not teaching basic academic and 
employability skills. (National Association of Manufacturers, 2001, p. 2) 

With regard to mathematics, the National Assessment of Educational Progress (NAEP) 
supports this criticism. The most recent results indicate that 37% of 12th-grade students 
performed at a “below basic” level on the math portion of the test. An additional 45% performed 
at a “basic” level, and only 18% were “proficient” or above (U.S. Department of Education, 
2004). What is more, there was very little improvement from 1990 to 2000. Analyses of other 
data have found only 30% of all students complete the minimum courses recommended for 
college entrance, and nearly one half of postsecondary students require remedial coursework 
once they get to campus (Steen, 1999). 
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International comparisons of student performance also underscore the need to improve the 
math skills of American students. The TIMSS (Trends in International Mathematics and Science 
Study) tests, which were administered at the fourth and eighth grades in 43 countries, found 
American students in the middle of the distributions in both grades when compared to countries 
at similar levels of economic development (Gonzales et ah, 2004). On the 2003 PISA 
(Programme for International Student Assessment) test, the math scores of 15-year-olds in the 
United States ranked 25th among 40 industrialized countries (Organisation for Economic Co- 
operation and Development, 2004). 

This report describes a pilot study of experimental research that develops and tests a model 
to improve the math skills of United States high school students through courses with work- 
related contexts. The courses involved in the study used to be called vocational education, and 
are now referred to as career and technical education (CTE).' Nearly all high school students take 
at least one CTE course during their high school experience, although not all of these are 
intended as preparation for employment (Silverberg, Warner, Long, & Goodwin, 2004). Some 
CTE courses provide opportunities for occupational exploration. Others assist students to acquire 
the skills they need to be informed consumers and effective members of their families and 
communities. Such courses are taught at both the middle- and high-school levels. Those CTE 
programs receiving funding under Perkins legislation must address the academic as well as 
technical achievement of CTE participants. 

This introductory section provides background on the types of students that CTE courses 
generally serve, the logic or rationale behind this pilot study, and the reason for its timeliness in 
the overall national education reform agenda. Chapter 2 provides a more in-depth discussion of 
the theoretical underpinnings of the research model. Chapter 3 describes the research methods 
and procedures. Chapter 4 describes the quantitative findings, and Chapter 5 the qualitative 
findings, and Chapter 6 offers an overall summary and conclusion. Short summaries are also 
provided at the end of each chapter. 



Background 

Mathematics remediation rates in postsecondary institutions have been high for decades 
(Hoyt & Sorensen, 2001; Rosenbaum, 1992). Surveys conducted in 1995 and 2000 found that 
about 4 out of 10 entering freshmen in public 2-year institutions and one fifth of those in public 
4-year institutions were enrolled in remedial courses (Parsad, Lewis, & Greene, 2003). The 
National Center for Education Statistics (NCES; 2001) found that 13.8% of beginning 
postsecondary students at 4-year institutions in 1995-1996 reported taking remedial courses in 
their 1st year, but the figure was 19.3% among students who had taken only a basic curriculum 
in high school. 

As noted earlier, nearly every high school student takes at least one CTE course during high 
school, and 43% take three or more courses that directly teach occupational skills — courses that 



' In 1999, the American Vocational Association changed its name to the Association for Career and Technical 
Education. Most state and local agencies have adopted the new name. In this report, the field will he referred to as 
career and technical education (CTE). 
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are labeled “specific labor market preparation” (SLMP) (Silverberg et ah, 2004). These SLMP 
courses are taught primarily in the 1 1th and 12th grades. Recent transcript analyses show that 
students who invest in these courses come disproportionately from groups that are at risk of not 
successfully completing high school (Levesque, 2003). More than 70% of youth in poor 
communities take three or more SLMP courses in high school, as do nearly 70% of Black 
students and 60% of Hispanic students. Students with disabilities, limited English proficiency, 
and low achievement are also overrepresented in CTE. In short, CTE serves large numbers of 
youth described in the discussion of closing the achievement gap. CTE enrollment occurs during 
the last 2 years of high school, during which time CTE students have likely stopped taking math 
and are preparing for transition to post-high-school work and education. 

Students who concentrate in CTE (defined as three or more SLMP courses in a coherent 
sequence) bring with them characteristics associated with low academic achievement, so it is no 
surprise that most national studies show that, as a group, CTE concentrators generally achieve 
lower scores than students in the academic curriculum on tests of cognitive ability (Oakes, 1985; 
Plank, 2001; Stone, 2002). Because the importance of mathematics has been ignored for students 
who are unlikely to continue their education at the postsecondary level, math requirements for 
them are typically quite low — at 2 years or less. Some analyses (e.g.. Stone & Aliaga, 2003) 
show CTE students to be taking more math than they have in the past, but still lagging behind 
college preparatory students in the number of algebra I, algebra II, and geometry courses taken 
(Delci & Stem, 1999). The combined result of these factors is that many young people who 
concentrate in SLMP courses in high school graduate with insufficient skills in math, reading 
and writing, and problem solving. 

The obvious solution to the problem — requiring more mathematics courses in high 
school — appears unlikely to overcome these deficiencies. Arguing for more rigorous math 
coursework are reports such as “The Condition of Education” (U.S. Department of Education, 

2003) , which found evidence that students from lower SES backgrounds (who are 
disproportionately represented in the “non-academic track”) improve their test performance more 
than their high-SES peers when they take more rigorous math. Yet, the simple solution of 
requiring more regular math courses during high school may not be prudent. As noted earlier, the 
NAEP assessments of mathematics shows a flat growth curve over the past 2 decades, during 
which time school districts across the nation have increased math and science coursework by 
four Carnegie units (see Levesque, 2003). Also during this time, the high school completion rate 
has been on a slow and steady decline (Swanson, 2004). These data suggest that doing more of 
the same is not an effective strategy for improving the math skills of high school students, and 
may in fact cause more youth to leave school before completion. 

Other approaches to engaging youth and improving academic performance may hold more 
promise. Some longitudinal surveys suggest that CTE increases educational engagement — 
especially for lower achieving high school students — and keeps them in high school through 
graduation (Plank, 2001; although other analyses show a neutral effect, e.g., Silverberg et ah, 

2004) . Case studies of three selected schools that are using CTE as a part of their whole school 
reform initiatives have found larger retention rates and evidence of improved math performance, 
compared with demographic ally and geographically similar control schools without such reforms 
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(Castellano, Stringfield, Stone, & Wayman, 2003). When they leave high school, those students 
who pursue CTE have demonstrable economic advantages over those who pursue traditional 
academic patterns and do not complete higher education (Bishop & Mane, 2004; Mane, 1999). 

The premise of this pilot study is that all of this points to an opportunity to improve the 
education of United States youth. A substantial proportion of students elect to invest in work- 
related CTE during the latter part of high school. Also, most of them will attempt college within 
the 1st 2 years following graduation (Silverberg et ah, 2004), but will encounter obstacles due to 
poor math skills (Rosenbaum, 2002). As will be discussed later in this report, CTE curricula are 
rich in math, and math is fundamental to almost all occupational areas. The hypothesis is, then, 
that enhanced mathematics in an occupational context will improve students’ math achievement 
while continuing to engage them in school and prepare them to reap future economic benefits. 
The strategy for testing this hypothesis follows in the next section. 

The Central Problem 

CTE courses inherently provide contexts for “applied” or “experiential” learning (Owens & 
Smith, 2000; Rogers, 1969). Applied learning is the delivery of content-area curricula within a 
relevant, authentic, and presumably more motivating context. Even though many CTE fields use 
mathematics to solve workplace problems, the math that students learn in such “applied” or 
“situated” contexts does not appear to transfer to paper-and-pencil tests. The mathematics in the 
CTE curriculum is implicit, both to the teachers and to the students; the challenge is to make it 
explicit, and measure the difference in meaningful outcomes. 

Many of the academic skills that are required for both workplace success and entry into 
higher education — skills like algebra and other mathematical concepts — are taught late in middle 
school and early in high school (Rosenbaum, 1992). Typically, there is little follow-up or 
reinforcement of these basic skills for nonbaccalaureate-degree-bound students in their later high 
school years. As a result, most CTE students are not exposed to the higher levels of math they 
will eventually need after graduation. How can the math skills of these students be enhanced 
during this critical juncture without detracting from the CTE skill-building they will need for the 
workplace? 

To answer this question, this pilot study proposed a model for enhancing math instruction in 
an occupational context. Rather than forcing math into the curriculum for a particular SLMP 
course, the model started with the principle that the math content ought to emerge from the 
occupational content. CTE teachers were asked to identify math concepts inherent in their 
curricula, and to move from specific occupational applications of this math to the broader 
mathematical principles that these applications involve. The goal was for students to be able to 
recognize how they could use mathematics to carry out necessary calculations, solve practical 
problems in their occupational areas, and transfer their math skills learned through CTE to novel 
or other contexts, while not negatively affecting their acquisition of technical knowledge in the 
CTE course. 
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This report describes the conduct and findings of a 1 -semester pilot study that tested the 
viability and effectiveness of this model. This pilot study, which was conducted in the 2nd half 
of the 2003-2004 school year, was designed to determine the feasibility of enhancing high 
school CTE courses in order to build skills in critical academic areas such as mathematics. Using 
cognitive and authentic assessments, the pilot study sought to test the basic hypothesis that high 
school students in a contextual, math-enhanced CTE curriculum will develop a deeper and more 
sustained understanding of mathematical concepts than those students who participate in the 
traditional CTE curriculum. 

If the hypothesis is supported, students taking part in the experimental curriculum will be 
able to transfer their applied math learning not only to novel settings in their technical field, but 
also to high-stakes paper- and-pencil tests and college entrance placement exams. Better 
performance on these tests in the experimental group will indicate that a math-enhanced CTE 
curriculum can provide a concrete demonstration of greater skill levels, thereby reducing the 
need for postsecondary remediation in math — of interest to both potential employers and higher 
education institutions. 

Implementing the pilot study required both curriculum enhancement and professional 
development for those teachers who taught enhanced curricula. The study also employed mixed 
methods of data collection and analysis (Creswell, 2002; Wenglinsky, 2002) to answer these 
research questions: 

1. Does a math-enhanced CTE curriculum improve the transferability of students’ math 
skills to traditional and applied tests of math knowledge? 

2. Does an enhanced CTE curriculum affect students’ likelihood of requiring postsecondary 
math remediation? 

3. Does enhancing a CTE curriculum reduce the acquisition of technical skills or 
knowledge? 

The methods used to address these questions are described in Chapter 3. It should be noted 
that the study’s perspective is that of CTE researchers, not mathematics educators. The purpose 
was to test the claim of CTE teachers that their students often learn concepts, especially 
mathematical concepts, that they do not grasp in the traditional academic classroom. Almost all 
classroom instructors have their favorite “aha” stories of students who finally understood abstract 
concepts when they saw them applied to real situations. Such stories are numerous, but no strong 
evidence exists to support the claim. 

The pedagogy and professional development for the study incorporated the advice of 
mathematics educators, but the primary purpose was not to advance instructional theory. It was 
instead to conduct a rigorous test of the claims of CTE classroom teachers. The approach was 
intended to test whether explicit mathematical instruction in an occupational context yields 
learning that affects performance on standardized tests of mathematical performance. 
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Supporting the Federal Education Agenda 

On the eve of the federal government’s reauthorization of Perkins III, in the wake of the new 
accountability guidelines in the No Child Left Behind Act of 2001, and in the face of the U.S. 
Department of Education’s Mathematics and Science Initiative (2003), many are debating the 
value of CTE. It is important that educators, policy makers, and the general public understand the 
potential for CTE to build the academic skills of students and provide added value to the high 
school experience of adolescents. Hull (2003), president of the CORD organization, states the 
case as follows: 

I am convinced that the new system [of CTE] could benefit any student, but 
especially students in the middle 60 percent..., [whom Parnell (1985) labeled] the 
neglected majority. Most ‘neglected majority’ students are far more capable than 
they are able to demonstrate in settings in which conventional, lecture-style 
teaching is the rule. 

Mathematical concepts are embedded in almost every CTE program, but there is no evidence 
to suggest that this situated math transfers to the workplace, other educational settings, or real- 
world problems. In CTE, students learn their trade in the context of actual work problems, and 
they perform best in areas where their learning can be applied, because they tend to be 
kinesthetic rather than audio-visual learners (see Orr, Thompson, & Thompson, 1999; Slaats, 
Lodewijks, & van der Saden, 1999). According to Kolb, less than 25% of students are abstract 
learners (CORD, 1999). Eor the remaining 75% of students, enhancing math in the CTE 
classroom can provide a valuable learning opportunity. Given that employers are now looking 
for workers with more math skills, it is just as important for CTE students to learn such “basic” 
academics as it is for them to learn skills and concepts directly related to their chosen fields. Eor 
a variety of reasons, moreover. United States high school students need to be able to apply their 
math knowledge to paper- and-pencil tests, which are critical for school funding at the state level 
under the No Child Left Behind Act. 

The pilot study described here was funded by the Office of Vocational and Adult Education, 
whose new vision for high schools supports and extends the goals of No Child Left Behind. 
Bindings that demonstrate how an academically enhanced CTE curriculum can build skill in key 
academic areas and add to the school engagement and long-term economic benefits of CTE (e.g.. 
Bishop & Mane, 2004; Plank, 2001) could aid in shaping future federal educational policy — 
especially in expanding opportunities for CTE participation in our secondary schools. In 
addition, such a finding would provide research-based evidence for the design of future programs 
that support the administration’s goal that every youth will complete high school with the 
academic knowledge and skills needed to make a successful transition into postsecondary 
education or training without needing remediation. 
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CHAPTER 2: TEACHING MATH IN CTE 

New technology continues to change the nature of work. As available jobs change, so do the 
skill requirements necessary for obtaining stable and well-paying work. The guidelines of 
traditional instructional design serve neither the students nor the workplace of today, which is 
filled with complex problems and rapidly changing environments and technologies (Derry & 
Lesgold, 1996). Indeed, expert performance is characterized by an ability to adapt ones’ skills to 
novel situations and actively solve problems (Ericsson & Charness, 1994). “The ability to cope 
with the non-routine is perhaps the only knowledge worthy of instructional design in many cases, 
since most of the rest can be acquired quickly from on-the-job performance” (Derry & Lesgold, 
p. 791). 

The National Alliance for Business, along with other associations, has examined the skills 
needed to obtain jobs that offer reasonable pay and opportunities for advancement and that are 
likely to experience growth (American Diploma Project, 2004). In an iterative, collaborative 
process, the Alliance worked in conjunction with Educational Testing Services (ETS) and 
employers to draft a list of general workplace requirements in English and mathematics. 
Employers emphasized a need for students to have experience with the application of skills. 
While they tended to value hands-on experience over classroom knowledge, employers stressed 
that algebra I skills, such as the ability to express problems as equations, and fluency with 
fractions and decimals, are essential in many workplaces. Several employers felt that the ability 
to work with equations with multiple variables was important. Eamiliarity with geometric 
formulas and ability to work with probability and statistics were also sought, as were data 
interpretation skills such as the ability to understand graphs and charts. Knowing how to problem 
solve in mathematics enhances an individual's ability to “function in the context of everyday 
situations and work settings” (Bottge & Hasselbring, 1993, p. 556). 

Another study (Bragg, 1997) measured how employers rate the various outcomes of 
education. On a scale from 1 to 5, with 5 representing very high priority, employers rated the 
ability to use algebra and geometry to solve workplace problems as the most important math or 
science skill (average rating 4.26). Other important math-related skills included using charts and 
graphs (3.68), and applying logical reasoning (3.89). 

These studies underscore the critical role of math-savvy individuals in the workplace. Higher 
math skills have been associated with higher rates of employment, promotion, and pay. This 
research has noted the importance of math in general, but additional literature asserts that skills 
taught in algebra I are particularly essential to individuals, organizations, and the nation. 

Mathematics in Today’s Schools 

The National Council of Teachers of Mathematics (NCTM, 2000) outlines math education 
goals commonly used throughout the country in their book Principles and Standards for School 
Mathematics. This national collaboration of math teachers states a goal of primary education: 
that all students will learn a rigorous core of mathematics that prepares them for work or 
postsecondary education. Traditionally, math has been divided into several tracks in which a 
minority of students receive a college preparatory math curriculum, while the remainder take 
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general, less-rigorous math classes. Those in the academically challenging classes tend to be 
White, male, and middle class (Oakes, 1985). This kind of tracking occurs, although 88% of 
eighth graders across demographic lines report a desire to attend college (Venezia & Kirst, 

2003). Schoenfeld (2002) asserted that poor and minority students continue to be systematically 
short-changed in math; similarly, the NCTM reports that female, minority, disabled, and non- 
English-speaking students are disproportionately the victims of low math expectations. Students 
who do not master mathematical skills in high school will need to learn remedial math skills in 
order to obtain decent jobs. If individuals do not have the opportunity to master complex 
mathematical concepts, their lack of skills will result in an under-optimized workforce. 

Algebra I skills are not simply an extension of arithmetic, but a separate form of mathematics 
that serves as a gatekeeper to advanced mathematic. Telese (2000) reports that algebra I is a 
prerequisite to all advanced math courses, and passing the course leads to higher scores on 
standardized tests, which in turn affects a student’s entry into many colleges. With the 
prevalence of high-stakes and other standardized tests in the country today, teachers may feel 
pressured to teach only those topics likely to appear on the tests. This may lead to a superficial 
level of algebra instruction, especially when coupled with a lack of preparation on the part of the 
teacher (Viadero, 2005). According to Telese, this is especially common in inner city and 
impoverished schools — schools that disproportionately serve minority students. Students who 
lack a fundamental understanding of algebra or possess only a formulaic understanding of the 
course will struggle with applying the formulas in a testing environment, thereby differentially 
affecting this group’s graduation and college entrance rates. 

Enrollment in algebra and advanced math courses, as well as alignment of the test to the 
course content, has been found to predict postsecondary student success (Venezia & Kirst, 2003). 
The researchers found that, although over 80% of eighth-grade students report wanting to attend 
a postsecondary institution, far fewer take the recommended courses to prepare for college or the 
admission tests. If students do take college entrance exams, they often confront a confusing array 
of tests that are poorly aligned in the transition from high school to college. More specifically, 
algebra I concepts are the highest level of math common on college admission tests; but once 
accepted to college, students may encounter placement tests that cover topics through algebra II 
and trigonometry. In addition, while contextualized and realistic math problems are common on 
high-school-level standardized tests, they are rare in college admissions or placement tests. 

One key predictor of postsecondary academic success is high-level-math coursetaking in 
high school. This is especially true for Black and Latino students (Venezia & Kirst, 2003). At 
least 80% of students who take calculus in high school graduate from a 4-year college, while 
only 8% of those who take only algebra I do. Because students who attend community colleges 
rarely take college-prep coursework, it is these students and institutions that are most likely to be 
penalized because of low math enrollments in high school. Community college students may find 
themselves taking multiple remedial math classes (which earn no credit) before they can take 
college-level course work; this costs the student and the institution in terms of both money and 
time (Venezia & Kirst, 2003). 
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In response to research with findings similar to those cited above, school districts have 
responded by making algebra I compulsory. As part of its Equity 2000 initiative, Milwaukee 
Public Schools (MPS) required all of its ninth graders to take algebra I, with the exception of 
those who had already completed the class or whose Individual Education Plans recommended 
an exemption (Ham & Walker, 1999). As a result, algebra enrollment went from 31% to 99% in 
1997. During the same period, algebra-passing rates for ninth graders went from 25% to 55%. 
Still, an average of 47% of ninth graders failed algebra I over the course of the period in which 
MPS implemented its Equity 2000 initiative. This indicates that dramatically more students are 
capable of succeeding in algebra I than would enroll in the class on their own; however, the high 
failure rate may be indicative of larger issues surrounding student preparation, class instruction, 
and curriculum design (Ham & Walker). The authors attributed the MPS algebra I failure rate 
primarily to low attendance, as 25% of students were absent on an average day. Teachers 
participating in follow-up focus groups and interviews cited two potential influences on algebra I 
failure rates: poor preparation during the middle school years, and poor preparation of the middle 
and elementary school teachers in charge of teaching math (Ham & Walker). Interestingly, at no 
point did the researchers or the focus group teachers suggest that current instruction may be 
suspect. 

As cited above, NCTM’s Principles and Standards for School Mathematics makes it clear 
that wanting all students to learn math does not mean that all students can or should learn math in 
the same way. One possible solution is to develop alternatives to the traditional algebra I course 
that do not sacrifice the rigor of the current program, but are more accessible to those students 
who are failing in the current program. 

Varying the curriculum design may be one way to promote achievement in algebra I. 
Research has shown that disengagement or lack of interest is a factor in low student achievement 
(NCTM, 2000). Students may disengage from math because of difficulty with the subject, lack of 
support, or simply boredom. Other students believe that the math they learn in school is not 
relevant to life after high school (NCTM). 

Impact of Math Instruction 

Data from the National Longitudinal Survey of Youth 1997 shows that students, including 
both academic and CTE concentrators, have been consistently taking an increasing number of 
credits in advanced coursework (Stone, 2002). Despite the push from state standards and college 
entrance requirements to raise math course requirements, however, the increase in math course 
taking does not seem to be having the desired effect: NAEP test scores have remained relatively 
flat over the course of the past 20 years, especially for 17-year-olds (NCES, 1999; although the 
largest average gain — 4 points — was significant when comparing test scores prior to 1990 with 
2003 scores). Given that the most positive results show a 4-point gain with 10 years of 
investment, these data would suggest that educators and policy makers would be well- served by 
looking to new techniques to raise students’ math test scores. 

While more students are taking advanced coursework, data from the National Center for 
Educational Statistics (NCES, 1996) still showed that about 25% of students graduating in 1994 
took fewer than 2 years of math. If students take their math requirements upon high school 
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entrance, it can be deduced that a quarter of students do not take any math past 10th grade. 
Therefore, many upper-level students taking graduation and college placement exams in their 
junior and senior years may not have been enrolled in a math class for 1 to 2 years prior to the 
examination. Even if students are enrolling in math in the 1 1th and 12th grades, improved math 
performance does not seem to have resulted. In 1994, 21% of 17-year-olds performed at or 
below modal level in math (NCES) — a percentile that has steadily increased since 1978. 

This downward spiral in mathematics performance continues into 12th grade and beyond. 

The TIMSS reports that United States 12th-grade students score about 50 points below the 
international average in advanced math, including numbers and equations, calculus, and 
geometry. There is, however, a chance to catch up on these skills at the postsecondary level: 71% 
of 2- and 4-year postsecondary schools offer remedial math courses, with an average 22% 
freshman enrollment rate (Levesque, 2003). American high school juniors and seniors are clearly 
unprepared for the math they will need after they graduate. 

Where can students get additional instruction in math during the critical 1 1th and 12th grades 
in order to fill this educational chasm? Students are more likely to be enrolled in CTE classes 
than academic classes in the 12th grade, although many students take CTE classes throughout 
their high school careers. Of the 98% of 1998 public high school graduates who took at least one 
CTE class, 36% of CTE credits were earned in the student’s senior year. In I Ith grade, students 
take 24% of their total CTE credits, while in 9th and 10th grades, students take 20% and 21% of 
their credits, respectively (Levesque, 2003). 

The gap created by low enrollment rates in math by upper-class students can potentially be 
filled by math embedded within CTE coursework. This could be accomplished by highlighting 
math content for students within the CTE classroom. Within each specific labor market 
preparatory (SLMP) area, mathematics can be taught in the context of the occupation (CORD, 
1999). Eor example, horticulturalists use math skills to estimate the number of pots of various 
diameters that can fit in an area of a greenhouse. Many examples can be drawn from across CTE 
areas. 

However, since CTE educators are not trained to teach math, explicit math content, such as 
algebraic formulas, rarely makes it onto the blackboard. It should: Under the 1998 
reauthorization of the Carl D. Perkins Vocational and Applied Technology Education 
Amendments (Perkins III), CTE classes are responsible for increasing students’ academic 
performance. More specifically, Perkins III states that an indicator of CTE performance is 
“student attainment of challenging State established academic... proficiencies” (Sect 113 item 
2Ai). As such, CTE programs, and therefore CTE educators, are expected to yield student 
academic gains. CTE educators have an obligation to enhance math requirements wherever 
possible within their CTE curriculum. 
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Curriculum Integration 

Enter curriculum integration. Integrating academic and career and technical education is one 
of the major policy objectives of the Carl D. Perkins Vocational Education Act, expressed first in 
1984 and subsequently restated in the 1990 and 1998 reauthorizations. It is also a policy 
hallmark of the School-to-Work Opportunities Act (Hoachlander, 1999). Curriculum integration 
has developed over many years and will continue to develop. As of 2003, the Association for 
Supervision and Curriculum Development (ASCD) offers the following definition: 

Integration is a philosophy of teaching in which content is drawn from several 
subject areas to focus on a particular topic or theme. Rather than studying math or 
social studies in isolation, for example, a class might study a unit called “The 
Sea,” using math to calculate pressure at certain depths, and using social studies 
to understand why coastal and inland populations have different livelihoods 
(ASCD, 2003). 

This definition articulates the true intent of curriculum integration. Despite the federal 
mandate originating from the Perkins legislation for curriculum integration in the classroom, 
there is little integration found in CTE or academic courses (Hoachlander, 1999). Integrated 
learning is more than simply connecting two or more disciplines in a way that their individual 
identities are maintained (Beane, 1993). Knowledge integration refers to students using previous 
knowledge and experiences as their foundation while exploring new dimensions of learning 
(Beane, 1997). Hoachlander goes further by identifying four types of integration: course-level, 
cross-curriculum, programmatic, and school-wide. Hoachlander stresses that effective 
curriculum integration must be guided by the central purpose of increasing student achievement 
and begin by clearly specifying educational goals. Eurther, if well-conceived and effectively 
delivered, integration can benefit any student and teacher. 

Contextual Learning of Mathematics in CTE 

The approach used in this pilot study can be seen as an example of curriculum integration, 
which educators often use interchangeably with “contextual learning.” This model is contextual 
in that math learning occurs within a real-world or applied context. Berns, Erickson, and 
Klopfenstein (2000) define contextual learning as learning that involves students connecting of 
content with the context in which that content could be used. They emphasize this connection of 
bringing meaning to learning. Similarly, Karweit (1993) defines contextual learning as learning 
that is designed to support students’ activities and problem solving in ways that reflect the real- 
world nature of such tasks. The U.S. Department of Education’s Office of Vocational and Adult 
Education and the National School-to-Work Office defined contextual learning as “learning that 
motivates students to make connections between knowledge and its applications to their lives” as 
family members, citizens, and workers. 

According to the contextual learning perspective, educators play a major role in helping 
students find meaning in their education and make connections between what they are learning in 
the classroom and how that knowledge can be applied in the real world. The contextual 
mathematics approach requires that students become more actively engaged as learners and that 
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educators change the way they deliver content in order to produce enhanced thinking about and 
use of mathematics concepts. 

The opposite of integrated, authentic, and contextualized is decontextualized and abstract, 
where math is presented as disconnected from any application (Brown, Collins, & Duguid, 

1989). In the case of algebra, equations are presented as things to be solved, or symbols to be 
moved around, or graphs to be drawn, without any discussion of the real-life applications of the 
math (Kieran, 1990). Some math educators believe that students have a lot of trouble with 
learning algebra in a decontextualized way (Kieran, 1992). Eor many students, it is too abstract 
too quickly, and therefore does not make any sense. This issue is particularly acute with low 
achievers (Woodward & Montague, 2002). Perhaps more than other students, low achievers need 
an authentic lesson as a way to make sense of abstract mathematics. Attempts to contextualize 
the symbols in word problems may not be an adequate or effective fix for all learners, 

“especially for students with mild disabilities and at-risk students, who have few resources to 
guide their problem-solving performance” (Montague, 1992, cited in Jitendra & Xin, 1997, p. 
435). 

One problem with contextual learning, however, is that students may be unable to transfer the 
knowledge learned in one context or situation to another context or situation. According to Lave 
(1988; Lave & Wenger, 1991), learning is inherently embedded within the activity, context, and 
culture in which it occurs, i.e., it is situated in that time and place. Similarly, according to 
Karweit (1993, p. 54), “Knowledge is. . . dependent upon and embedded in the context and 
activity in which it is acquired and used.” Unless students are taught the abstract principle behind 
what they are learning in context and guided through other contextual examples, it is unlikely 
that cognitive transfer will occur outside the classroom. Bay (2000) suggests, “Teaching via 
problem-solving is teaching mathematics content in a problem-solving environment. Learning in 
this approach involves learning through a concrete problem and eventually moving to 
abstraction” (Bay, p. 3). The use of authentic situations serves to “anchor” the symbolic and 
abstract math in situations that are familiar and real to students, which serves to help them make 
sense of the content (Cognition and Technology Group at Vanderbilt, 1990). 

Many educators and researchers believe that in order for students to make links between 
concepts, they need to go through a process or series of steps, beginning with an introduction to 
solving a real, relevant problem; practicing on several similar examples; and then applying the 
concept learned to a more abstract problem. Basic skills can be learned by rote, but the more 
flexible knowledge needed to become skilled requires “deliberate” practice (Ericsson, et ah, 
1993). One possible way to create deliberate practice is by asking students to solve a problem 
repeatedly in ways different from those methods previously used. Anderson (1996) might 
describe the process of readdressing a problem as a tuning stage. Brownell has called this 
“meaningful habituation” (cited in Allen, 2003). According to the theories described in this 
section, learning problem solving in a real world context and practicing both similar and novel 
problems on a continuum from more contextualized to more abstract should pave the way for 
students to be able to transfer their skills to new situations and environments. 
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This study’s premise assumes that conceptual mathematics learning and transferability of 
skills can be enhanced by using a contextual, applied approach, and that testing students on both 
typical (abstract) and applied math problems will show whether this has been accomplished. It is 
assumed that the students participating in this study have already been introduced to algebraic 
and other procedural knowledge via their mathematics coursework during junior-high, freshman, 
and sophomore years. This model provides a framework for CTE teachers and their math-teacher 
partners to develop lessons that enhance the mathematics that naturally occurs in occupationally 
specific CTE courses. This is accomplished by making the concepts and procedures explicit 
using successive approximation — i.e., moving from the fully embedded example in successive 
waves toward less contextualized and more abstract examples of the math concept — using the 
seven elements of the National Research Center for Career and Technical Education (NRCCTE) 
model. (These enhanced lessons constitute the experimental intervention tested in the study.) 

This study will also address a gap in the literature on secondary school mathematics 
curricula. The 1989 NCTM Principles and Standards for School Mathematics announced a 
departure from rote memorization in math learning, and urged teachers to focus more on student 
engagement and realistic math problems. The summary chapter of a recently published book on 
research in classrooms using these standards-based mathematics curricula in the 1990s concludes 
that such an approach “works” (Kilpatrick, 2003). However, because of the difficulties in 
conducting curriculum research, to date there is no single yardstick or truly experimental design 
to conclusively show that the new approach works better than the traditional approach in raising 
student math achievement. This study will provide a set of common measures and a randomized 
experimental design to assess the effectiveness of a contextual math intervention. 

The NRCCTE Model 

The NRCCTE Math-in-CTE model, which was the experimental intervention in this study, 
involved both pedagogy and process. Without one or the other, there is no result. In 
mathematical terms: 



(Pedagogy)(Process) = Student Math Performance 

The NRCCTE model emerged from attempts to answer this question: How could we 
capitalize on students’ interest in CTE content to improve their understanding of mathematics to 
the point that it would influence their ability to use math in other contexts, including 
standardized tests? 

The basic assumption was that the mathematics taught in CTE courses should arise directly 
out its occupational content. The goal is for students to see math as an essential component of the 
CTE course content, like a tool — a saw, a wrench, or a thermometer — needed to successfully 
perform the tasks of the occupation. Therefore, it was essential to develop a model that CTE 
teachers could use to improve instruction in the math concepts embedded in their occupational 
curricula. Also, it was acknowledged that CTE teachers were not mathematics instructors and 
would need assistance in identifying the math in their curricula and developing lessons to teach 
it. 
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The Pedagogy 

As a part of the NRCCTE model, the research team and educational consultants created a 
pedagogical framework to guide the development and instruction of math-enhanced lesson plans. 
This framework was labeled the “seven elements of a math-enhanced lesson” (see Eigures 1, 2, 
and 3). Using lesson plans developed with these seven elements, teachers assisted students in 
making links between concepts through a process or series of steps that begins with an 
introduction to solving a real, relevant problem; proceeded with students practicing with several 
similar examples; and ended with students applying the concept they learned to a more abstract 
problem. Eigure 1 depicts these seven elements, and Eigure 3 explains each element in more 
depth. 
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Figure 1. The NRCCTE model — the seven elements of a math-enhanced lesson. 
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Figure 2. Sample building trades math-enhanced lesson — using the Pythagorean theorem. 
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Figure 3. The seven elements: components of a math-enhanced lesson. 

1. Recognizing math with your class (“Pull & Point”) 

When you come to the part of your lesson where predetermined math exists, verbally 
recognize the math . . .show students by “pulling out” and “pointing out” in the lesson, 
activity, project for the day. 

2. Assess students’ math awareness: 

Using suggested questions, evaluate how much students know about the math 
concept/skill being addressed. 

Questions: “What do you know about ?” Or 

“What can you tell me about ?” 

3. Walk through the “pulled out” math example: 

Walk students through the steps/processes needed to complete the example. 

Ask students to take the lead depending on level of understanding. 

4. “Enhance” the math in your lesson: 

a. Share the “generic” math principle/concepts with students. Purposely use math language 
and ask students to do so as well during the enhancement. 

b. The transition from CTE to math vocabulary should be gradual throughout the lesson, 
being sure to never completely abandon either set of vocabulary once it is introduced, e.g. 
use the term “slope” along with the term “pitch.” 

5. Reinforce the enhancement - Supply students with: 

a. similar math example(s) from a similar CTE scenario and 

b. generic math example(s) similar to those they might see in a math class or on a math test. 
(Students may work through the math principle or concept individually or in groups.) 

6. Check for Understanding -Ask students the following questions: 

“Can you explain the math step(s)/concept(s) that we used today?” 

“How would you explain these math steps/concepts to someone else?” 

7. Expand the Enhancement - Ask students to create: 

a. a math example within the CTE lesson context OR provide students another CTE 
scenario (which addresses the same math principle/concept) but with an error in logic and 
have them correct the work. 

b. a generic math example (similar to those they might see in math class or on a math test) 
OR provide students another generic math example (which addresses the same math 
principle/concept) but with an error in logic and have them correct the work. 

(Students should be allowed to and even encouraged to actually solve their homemade 
examples.) 
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The seven elements in the pedagogical framework have many parallels with Gagne’s The 
Conditions of Learning (1965) and Hunter’s ITIP (Instructional Theory Into Practice; 1982). 
What differentiates this framework from the aforementioned, however, is the increased emphasis 
on moving from specific applications to general principles using a hands-on approach. Elements 
3 and 4 move the instruction from the contextualized CTE problem to the traditional math that 
students are likely to encounter in standardized tests — a form of successive approximation. In 
Elements 5 and 6, the instruction moves to both contextualized and traditional examples to 
reinforce and expand application of the math. (In the pilot study, few teachers successfully 
accomplished Element 7. This led to modification of the seven elements for the full-year study. 
[See chapter 6.] As mentioned earlier, the creation of explicit connections between situations is 
critical if students are to transfer their knowledge and skills outside the classroom, whether it is 
to another context or to an abstract testing situation. Teachers following the NRCCTE 
framework made the math in their lesson explicit in order to promote a stronger linkage between 
what students learned in a particular project situation and the abstract concept behind it (National 
Research Council, 2000). Eor example, when using a T-square in an agricultural mechanics 
class, the teacher was encouraged to show the class the formula b^= C (i.e., the Pythagorean 
theorem). The assumption was that if the teacher modeled a meta-cognitive approach to problem 
solving (i.e., the process by which one consciously thinks back to similar problem situations), it 
was likely that students would think back to their lesson on the T-square in carpentry class when 
seeing a formula in a paper- and-pencil test, and it is likely that they would remember how to 
solve the problem. 

The seven elements also have many parallels with the assessment framework that underlies 
the Program for International Student Assessment (PISA) sponsored by the Organisation for 
Economic Co-operation and Development (OECD). In this framework, the term 
“mathematizing” is used to describe a process that involves translating a problem from “reality” 
into mathematics, and includes activities such as: 

• identifying the relevant mathematics with respect to a problem situated in reality; 

• representing the problem in different ways, including organizing it according to 
mathematical concepts and making appropriate assumptions; 

• understanding the relationship between the language of the problem and the symbolic and 
formal language needed to understand it mathematically; 

• finding regularities, relations, and patterns; 

• recognizing aspects that are isomorphic with known problems; 

• translating the problem into mathematics, i.e., to a mathematical model (OECD, 2003, p. 
39; cited by de Lange, 1987, p. 43) 

The OECD framework was used in the development of assessments for mathematical skills 
and knowledge, but OECD has not advanced the framework as a teaching model. Nevertheless, 
the seven elements, which were developed independently from the OECD framework, have 
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many similarities to it, and could be considered a framework for teaching the mathematizing 
process within defined occupational contexts. 

The Process 

The development of the pedagogical framework (the seven elements) was only one aspect of 
the NRCCTE model. The experimental intervention also required the creation of a process 
through which the CTE teachers could learn to develop and teach the math-enhanced lessons. 

The process included partnering CTE teachers with math teachers, building curriculum maps that 
intersected math concepts with CTE curricula, providing professional development for the 
teacher teams, and implementing the math-enhanced lessons. Each of these aspects of the 
process is described in more detail below. 

Establishing CTE-Math Teacher Partnerships. Recognizing that a majority of CTE teachers 
are not formally prepared to teach math, the pairing of CTE teachers with math-teacher partners 
was a critical component of the NRCCTE model. As part of the participant application 
procedure, the CTE teachers were required to identify math-teacher partners, preferably from 
their own schools, who were willing to work with them throughout the study. The role of the 
math-teacher partners was to help the CTE teachers identify the mathematics in their specific 
CTE courses, to assist the CTE teachers in developing the math-enhanced lessons, and to suggest 
instructional methods to highlight the mathematics concepts. Importantly, the role of the math 
partners was not to team teach or in any way teach the math /or the CTE teacher; instead, they 
were asked to provide math support to the CTE teacher prior to and after they taught their math- 
enhanced CTE lessons. 

Building Curriculum Maps. Curriculum mapping is a well-established procedure used by 
state education agencies (e.g., Colorado and Michigan departments of education), as well as by 
nonprofit curriculum development organizations (e.g., V-TECS, COMAP). This is also a strategy 
used by corporations to identify the academic content of jobs. In the study model, math experts 
and consultants were asked to begin the process by mapping the intersection of math concepts 
with the content of specific SLMP curricula and content standards (see Appendix A). Eor 
example, the use of proportions and ratios is critical to the preparation of medicines for the 
health occupations group. These maps were then provided to the CTE-math teacher teams to 
further revise and use as a basis for constructing math-enhanced lessons within their SLMP. 

Providing Professional Development. Individuals with expertise in teacher training and 
curriculum integration worked with the NRCCTE research team as consultants to plan and 
conduct the professional development workshops. CTE-math teacher teams attended all 
workshops together, and remained partners throughout the study. The goals of the workshops 
(each of the replication sites met separately and focused on their own occupational area, or 
SLMP) were as follows: 

• to ensure that the CTE-math teacher teams were able to work together 

• to have teacher teams revise the curriculum maps and subsequently identify math 
concepts common to their curricula 
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• to agree upon and develop a set of 5-10 math-enhanced lessons plans to implement in the 
classrooms 

• to guide and support CTE-math teacher teams in using the seven elements to develop 
math-enhanced lessons 

• to ensure that CTE teachers were equipped to implement the math-enhanced lessons 

Implementing the Math-Enhanced Lessons. The NRCCTE research team and educational 
consultants who assisted with the design of the NRCCTE model were sympathetic to the 
demanding context of high schools, and understood that implementation of the math-enhanced 
lessons could not simply be assumed. Therefore, participating teachers were asked to agree to the 
following expectations for implementation: 

• CTE teachers would teach the full set of math-enhanced lessons developed and agreed 
upon in the professional development workshops for their SLMP 

• CTE teachers would implement the math-enhanced lessons as an integral part of their 
curricula, teaching them where the math naturally occurred, as opposed to teaching them 
as stand-alone math lessons 

• math-teacher partners would provide ongoing support throughout the implementation 

• CTE teachers would teach the lessons on their own 

Summary 

This chapter presents a small sample of the research identifying the increasing importance of 
mathematics in the workplace. It then summarizes the arguments underlying the procedural and 
contextual, or integrated, approaches to the teaching of mathematics. The procedural stresses the 
importance of developing a sound foundation in basic operations. The contextual emphasizes the 
need to make mathematics usable by applying it to real problems. Components of both 
approaches have been incorporated into the seven elements developed for teaching the math 
inherent in CTE curricula. 

The NRCCTE model, however, is much more than the seven teaching elements. Essential to 
the model is ongoing teamwork between CTE instructors and their math partners to identify the 
math concepts embedded in the CTE curricula and to develop lessons to explicitly teach the math 
that they identify. These lessons should be designed to teach the math first in an occupational 
context, and then to generalize to the more traditional forms that students are likely to encounter 
on standardized tests. As the CTE instructors deliver these lessons, math partners should be 
available to provide continuing support. 
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CHAPTER 3: DESIGN AND METHOD 

This study was designed as a field experiment, with random assignment of teachers to the 
experimental and control conditions. The logic of a randomized design is that all the unmeasured 
factors that may affect performance on the dependent variable will be randomly assigned to the 
two groups. If a statistically significant difference is found between the groups, it can be 
attributed to the experimental intervention (Cook, 2002). The random assignment was tested to 
determine if it yielded groups with comparable math pretest results, as will be explained later. 

Eollowing the vision of No Child Left Behind and the study models being emphasized by the 
Institute for Education Sciences (formerly the Office of Educational Research and 
Improvement), teachers of students participating in occupationally specific career and technical 
education (CTE) courses were randomly assigned to either the experimental or control condition. 
The control, or counterfactual, condition permits the researcher to measure what would be 
expected if the students in the experimental classrooms had not received the intervention. 

All research design involves some element of compromise. This study conducted the random 
assignment at the classroom — not the individual student — level, and thus the unit of analysis is 
the classroom. Assignment was made by recruiting a pool of CTE teachers, and assigning them 
at random to one of two conditions: experimental or control. This strategy was pursued for two 
reasons. Eirst, it avoids the well-documented problems of parental opposition to such activities 
(for extended discussions of random assignment studies, see Cook, 2005; and Stern & Wing, 
2004). With classroom- level assignment, all students received or did not receive the treatment, 
and could only opt out of the testing regimen; very few opted out. The second reason is more 
important: because CTE classes are often “singletons” in their schools, there would have been no 
control class to which they could have been assigned. Even had there been an alternative section 
or class, the randomization process ensured that the control schools were, in fact, other schools. 
This was done to limit the crossover effect described by Cook. One consequence of this decision 
is that it limits the ability to engage in subgroup analyses using student- level characteristics. 
Because of the known characteristics of CTE participants, however (see previous chapters), any 
effect found from the intervention would provide benefit to those youth most in need of 
improved math performance. 

A second design decision was to conduct six simultaneous replications of the same study (see 
Eigure 4). 
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Figure 4. Six simultaneous replications of the experimental study. 

According to a government research office (Yang, 2002), replication means the repetition of 
treatments in an experiment. There are two reasons replications are needed: 

• If a treatment appears only once in an experiment (i.e., n = 1), there is no replication of 
the treatment, and the error associated with the estimate of the treatment effect cannot be 
estimated. Experimental error occurs when two or more identically treated experimental 
units fail to yield identical results. Thus, replication of treatments provides an estimate of 
experimental error. 

• Replication also enables us to obtain a more precise estimate of the main effect of any 
factor, since the standard deviation of the mean = oVn, where represents the true 
experimental error and n the number of replications. 

Schafer (2001) advocates for routine replications in field research since researchers have 
more limited control over experiments than in the laboratory. He argues: 

When results are consistent across several studies, there is a stronger basis for 
observed relationship(s) than the support that is available within each study by 
itself, since results that have been replicated are considered more likely to 
generalize (continue to be observed). It is also possible to compare the studies 
with each other to identify constructs that interact with, or moderate, 
relationships. Although these advantages exist whether or not the research 
includes experimental control, the opportunity to replicate a basic study design in 
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multiple field contexts is more likely to be available to the applied researcher and 
is a technique that can lead to stronger inferences in any setting. Thus, it is 
recommended that persons who conduct field research try to include replication as 
a fundamental feature in their studies. 

As Gueron (2000) observed, “Random assignment studies must be used judiciously and 
interpreted carefully to assure that they meet ethical norms and that their findings are correctly 
understood. Random assignment can answer the important ‘Does it make a difference?’ and ‘Eor 
whom?’ questions, but it must be combined with other approaches to get answers to the critical 
questions of ‘Why?’ and ‘Under what conditions?”’ (p. 1). It is for this reason that multiple 
settings were studied, and that several qualitative measures, designed to address the “why” 
question that will be described later in this report, were built in. 

Einally, the approach of multiple simultaneous replications was taken to address one of the 
key criticisms of experimental research in education. Critics argue that random- assignment 
studies are not only rare, but that researchers who conduct them typically evaluate high-quality 
programs that serve only a few children, often at a single site — making it hard to generalize 
findings to large-scale programs or more diverse populations of children (Magnuson & 
Waldfogel, 2005). 

The six occupational areas selected in this study represent the breadth of CTE programming. 
The study was replicated in a program (business and marketing) that is essentially classroom- 
based, a heavily skill-oriented occupational area (automotive technology), two occupational 
areas identified as high-tech and high-growth (health occupations and information technology), 
and two programs historically associated with CTE (agricultural mechanics and horticulture). To 
the extent that the findings are consistent across these replications, they can be generalized to 
most occupationally specific CTE programs. 

The experimental intervention, or treatment (described in detail in the procedure section), 
consisted of the development and delivery of math-enhanced lessons that were created by teams 
of math and CTE teachers, and delivered by the CTE teachers alone. The intervention was 
deliberately not team teaching.^ The lessons were designed to raise the embedded mathematics in 
the CTE curriculum to a level of explicit learning by incorporating direct instruction, modeling, 
application, and abstraction. The enhancements were intended to facilitate student mastery of 
math concepts and the ability to transfer that competency to novel settings. 

The dependent variable in the study was student math achievement, as measured by 
standardized tests. Because of random assignment, the math performance of the experimental 
and control groups could be directly compared, and any differences could be attributed to the 
treatment. To examine different types of math performance, student math achievement was 
measured by three different instruments: 



^ The CTE students or teachers were not to feel that an outside “math expert” was coming in to teach the math. The 
CTE classroom was to function on its own as a place where math could be learned, and used as an integral part of 
the curriculum — not as an add-on. 
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• a traditional measure of math performance (TerraNova) 

• a problem-based, applied, measure of math performance (WorkKeys) 

• a widely used college math placement exam (ACCUPLACER) 

To determine if the instructional time devoted to math had a negative impact on the learning 
of occupational skills, tests of technical skills and knowledge, appropriate for each occupational 
area, were also administered (these will be described in more detail in the measures section). 
Students in each classroom took one of the three math tests (thus there is a classroom-level score 
for each math test, but individual students were not overburdened by having to take all three); all 
students took the skills test. 

To ensure that this study would contribute to understanding strategies for improving math 
skills in high school students, an advisory committee of methodologists was formed (see 
Appendix B). This group of experts met twice before the start of the project to discuss details of 
the theory, methods, design, and implementation of the research. Eurthermore, the team of 
researchers across the multiple replication sites included CTE teacher-educators, math experts, 
educational psychologists, curriculum integration specialists and administrators, and both current 
and former secondary CTE and math teachers (see acknowledgments section). This research 
team met via conference call once per month during the duration of the study, as well as twice in 
person at the NRCCTE headquarters. 



Procedure 

A study of this magnitude required the cooperation of national organizations involved in 
CTE, many local- and state-level administrators, and university researchers. This section 
describes the efforts made by NRCCTE and the replication sites to recruit and prepare teachers. 

Recruitment 

Most commentators on large, national, experimental studies suggest that recruiting schools 
and establishing an experiment takes 3 years (Cook, 2002; Whitehurst, 2003). However, for this 
pilot study, teacher participants were recruited and experimental sites established within 6 
months. This was partly due to the fact that entire districts or schools did not have to be 
recruited; and the unit of analysis was the teacher (i.e., aggregate class performance). Recruiting 
individual teachers was an approach that allowed us to build a large enough sample within the 
time constraints of the study. The study’s statistical expert suggested control and experimental 
samples of at least 20 and 20 in each replication in order to be able to detect any effects of the 
treatment. After recruiting teachers at each site, school/district permission for each participating 
teacher was obtained, rather than seeking permission before recruiting, which may have taken 
much longer. 

Initial recruitment occurred through four principal sources: the Association of Career and 
Technical Education, the National Council of Teachers of Mathematics, The National 
Association for Tech Prep Leadership, and SkillsUSA. A recruitment letter was sent to teachers 
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on mailing lists obtained from these organizations announcing a “program” to enhance CTE 
courses with more rigorous and explicit mathematics lessons. Interested teachers were given an 
application or directed to log onto the study’s Web site to download an application to participate, 
which they could mail, e-mail, or fax to the NRCCTE. 

This mass-mailing strategy was expected to yield a national pool from which to first select 
states with the largest number of interested teachers in the same occupational areas for inclusion 
in the study, and then to conduct the random assignment. Due to a late start to the project and a 
conflict of proposed summer workshop dates, the sample generated by this initial mailing was 
much lower than anticipated, and the needed sample of 40 teachers (20 experimental, 20 control; 
identified through a power analysis) within individual states was not achieved. As a result, each 
of the NRCCTE partner institutions undertook a more focused recruitment of teachers in their 
own states and, in some cases, neighboring states. 

The efforts at the sites drew upon personal relationships and professional associations within 
the targeted SLMPs. Direct mail and e-mail were used to recruit CTE teachers at Site A^. To 
support the direct mail and e-mail, an associate director for the state department of education and 
the director of K-16 initiatives for the Board of Regents sent e-mails to all high school 
principals, career center directors, and Tech Prep coordinators to inform them about the study 
and to ask them to encourage their teachers to participate. The staff at Site A made personal 
presentations about the study to three summer workshops for Tech Prep instructors. When these 
measures yielded a low response, telephone calls were made to all teachers in the occupational 
area in one state to request their participation. 

Site B focused on teachers who taught programs that were certified by an industry-sponsored 
accreditation board for that specific type of CTE program. To start the recruitment, the board’s 
manager for one national region sent an e-mail to the managers for all states in the region, asking 
them to encourage the teachers in their states to participate. The site staff followed up with these 
state-level managers and provided them with applications for all teachers who expressed interest. 

At Site C, the state supervisor for the occupational area was instrumental in the recruitment 
of teachers, by sending a letter of support that strongly encouraged teachers to participate. One of 
the project’s staff members in the state developed a list of 25 teachers/schools considered likely 
to participate, and contacted them personally. A presentation about the study during a summer 
conference for CTE teachers by the NRCCTE Director yielded a number of applications. 

Another presentation by the NRCCTE Director at a conference of CTE educators, at Site E, 
helped to increase the pool of interested teachers, which study staff in that state had previously 
recruited. The two study staff members in that state had focused their initial efforts on the school 
districts of a major city and its western suburbs, as they both had a history of involvement in the 
school district. They also worked with professional associations and made personal telephone 
calls to local CTE directors to inform them of the study and to ask them to encourage their 
teachers to apply. 



^ Participants in this study were promised anonymity regarding the outcomes at their individual sites. To provide this 
anonymity, the six separate sites are referred to hy capital letters (A to F). 
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Staff from Site E met with CTE instructors at the district conferences in five administrative 
districts of the state. A presentation was made that described the proposed study. Teachers who 
expressed an interest were given an application. To reach a pool of 40 interested teachers, 
follow-up telephone calls were made to selected teachers, based on personal recommendations of 
program specialists for the five administrative districts. 

Site D followed a procedure similar to that of Site E. Staff made presentations about the 
study to conferences of CTE teachers to request their participation. Applications were distributed 
to interested parties and follow-up e-mails and telephone calls were made to encourage 
application submissions. 

The application that teachers completed required their signatures, as well as those of their 
school administrators, to indicate that the teachers’ participation in the study would be supported. 
The application also called for the identification of a math teacher who would partner with the 
CTE teacher if they were selected for the experimental group."^ 

At least 40 teachers were recruited in five of six SLMPs, but in one, only 30 completed the 
full application process. (In addition, some of the original teachers who applied dropped out after 
they were assigned to a group.) The nature of the design meant that self- selection bias by 
teachers was inevitable. Given the concern for the protection of human subjects, which requires 
voluntary informed consent, self-selection exists in virtually all research projects. The teachers 
who volunteered to participate likely shared more measurable and immeasurable attributes than 
teachers who did not, which might include a higher level of comfort in teaching mathematics. 
This is a limitation to generalization of the findings, but not to the validity of the group 
comparisons, because the assignment process distributed self-selected attributes to both groups 
randomly. 

Table 1 shows the number of teachers who volunteered, the number assigned to the two 
conditions, and the number who actually participated in the study. In each replication site except 
one, at least 20 CTE teachers and their math-teacher partners were randomly assigned to the 
experimental group, and the other 20 were assigned to the control group. This design yielded 236 
CTE teachers, 104 math teachers, and 3,950 students across 12 states. 



CTE teachers were allowed to apply without math-teacher partners, hut the most successful CT-math partnerships 
were the ones that were chosen hy the teachers themselves. 
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Table 1. 

Number of Teachers Who Completed Applications, Assignment to Conditions, and Participation 
in the Study 



SLMP Site 


Applied 


Experimental 


Control 


Participated 


Withdrew 


Participated 


Withdrew 


A 


30 


8 


8 


13 


1 


B 


50 


22 


4 


22 


2 


C 


60 


28 


3 


27 


2 


D 


45 


20 


3 


18 


4 


E 


46 


18 


5 


22 


1 


E 


43 


18 


4 


20 


1 


Total 


274 


114 


27 


122 


11 



Both groups of teachers were sent letters by NRCCTE informing them of their selection. The 
experimental group teachers were told that they would implement a math-enhanced CTE 
curriculum during spring 2004 that they would help develop with their math-teacher partners at 
professional development workshops in fall 2003. The control group was asked to implement 
their traditional curriculum “as usual.” The local sites followed up with letters to the 
experimental teachers with details about the workshops and a request for copies of the 
curriculum materials they had used in the past. Control group teachers were also asked for copies 
of their curriculum materials; this was to document their teaching of the course before their 
involvement in the study. Teachers from each SLMP were randomly selected for an in-depth 
interview about their teaching practices (see data collection section). 

As an incentive for participation, both CTE and math teachers in the experimental group 
were told they would each receive a $1,500 stipend at the end of the study, plus all costs for 
travel, food, and lodging to attend the workshops. This stipend amount was originally established 
and publicized for a year-long study (the original plan). However, funding approval was delayed 
and the experiment was reduced in scope from 1 full academic year to 1 semester. Because many 
of the teachers had signed on expecting a $1,500 stipend, that level of compensation had to be 
maintained despite the reduced scope of the project. Experimental teachers also had the option of 
receiving continuing education units or college credit through one of the partner institutions. 

CTE teachers in the control group were told they would receive a $500 stipend for the 
classroom time used for testing; they were also given the option of receiving professional 
development in the math enhancement of their CTE curriculum in the summer of 2004 
(subsequently postponed to 2005, due to the extension of the study to 1 full year in 2004-2005). 

Erom the total of 274 complete applications that were received, 141 teachers were randomly 
assigned to the experimental and 133 to the control groups. Thirty-eight of those assigned did not 
participate (27 experimental and 11 control). All of these were “up-front” dropouts; they 
withdrew before any participation in the project. The higher number of dropouts from the 
experimental group was primarily due to the requirement that these teachers attend the 
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professional development workshops that are discussed below. Many had scheduling conflicts, 
and therefore decided not to participate. 

Attrition did not result in the schools of the participating teachers having characteristics 
different from the schools of the teachers who withdrew. The characteristics of both sets of 
schools were compared using data from Common Core of Data compiled by the U.S. Department 
of Education’s National Center for Education Statistics, and found no significant differences. 

(Eor the details of these comparisons, see Appendix C.) 

Parents of students in the study, both experimental and control, received a letter informing 
them of the nature of the study. They were asked to sign and return a form only if they did not 
want their child to participate. Very few parents returned the form. Students whose parents did 
not object to their participation signed assent forms before the surveys and tests were conducted. 
To encourage their participation and involvement in the data collection, students were given a 
$10 gift certificate for each of the two survey /test administrations (one “pre” and one “post”). 

Gender and race/ethnic characteristics of students in the experimental and control groups 
who reported these characteristics are shown in Table 2. Of the 12 comparisons of experimental 
and control group students, only one is significantly different: the race/ethnicity category in Site 
A.^ In this SLMP, almost 80% of the control group reported themselves as non-Hispanic 
European/ Anglo, while less than 50% of the experimental group was in this category. Additional 
analyses comparing the two groups on six other characteristics are shown in Appendix C. 

Overall, the characteristics of the students in the groups were quite similar. 

Table 3. 

Gender and Racial/Ethnic Characteristics of Experimental and Control Students by Site 



SLMP 

Site 


Male 


Non-Hispanic 
European/ Anglo 


Experimental 


Control 


Experimental 


Control 


% 


n 


% 


n 


% 


n 


% 


n 


A 


75.6 


135 


75.9 


166 


44.4 


124 


78.7 


155 


B 


93.4 


351 


95.7 


396 


76.0 


317 


73.9 


357 


C 


10.6 


432 


11.2 


446 


61.6 


375 


60.6 


421 


D 


51.4 


420 


50.3 


314 


72.8 


389 


65.8 


295 


E 


55.5 


357 


50.3 


356 


67.3 


337 


66.7 


342 


E 


82.4 


204 


87.7 


243 


62.6 


179 


65.1 


232 


Total 


55.7 


1899 


57.5 


1921 


65.8 


1721 


67.4 


1802 



Note, n is the base for the percentage (e.g., the 75.6% experimental males in Site A is based on a total of 135 
experimental participants who reported their gender in that SLMP). Ns for gender and «s for racial/ethnic 
characteristics differ because of missing data for some students. The distributions of these characteristics within 
SLMPs differ significantly between the experimental and control groups only for racial/ethnic background in Site A. 
Appendix C shows the percentages for each of the major racial/ethnic categories. 



^ Since the analysis was at the classroom level, race/ethnicity could not be controlled for. An analysis at the student 
level, however, showed no effect of race/ethnicity on posttest scores when controlling for pretest. 
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These data on the characteristics of the students in the experimental and control groups are 
presented to demonstrate that random assignment at the classroom level yielded similar, but not 
fully equivalent, groups. Because there were some minor differences, the pretest was used as a 
covariate to analyze posttest data. These analyses are at the classroom level, and thus satisfy the 
statistical assumption of random assignment to conditions. Because analyses are at the classroom 
level, no analyses were conducted by major subgroups (e.g., race/ethnicity, gender). 

In order to keep students anonymous, to minimize teacher influence on student participation, 
and to facilitate simultaneous testing dates at multiple schools, each site identified a liaison at 
each school to administer the surveys and tests. Liaisons could be a counselor, principal, 
secretary, or retired teacher; some individuals acted as liaisons for multiple schools within the 
site. The role of the liaison was to act as a link between the researchers and the schools; they 
were responsible for scheduling test dates and arranging data collection. They were also 
responsible for explaining the study to students and distributing consent forms in the control 
classrooms (experimental teachers were responsible for these tasks in their classrooms). Liaisons 
were also responsible for assigning ID numbers to students and keeping all teacher and student 
data anonymous and confidential. Each liaison signed a confidentiality agreement in order to 
participate. Lor their duties, liaisons were each compensated with a $250 stipend. 

Implementing the NRCCTE Model 

The NRCCTE model, presented fully in Chapter 2, involved both pedagogy and process. In 
this section, implementation of the pedagogical framework and the processes are described in 
more detail. 

To prepare the CTE-math teacher teams to function collaboratively in the experimental 
effort, each of the replication sites conducted professional development workshops in the fall of 
2003, preceding the start of the experimental treatment in January 2004. Workshop facilitators 
met in late summer 2003 to jointly plan the workshop agendas, ensuring that all experimental 
teachers received the same professional development. The agendas that emerged from this 
process included strategies for effective team building and guidance on how to work together to 
raise the mathematics embedded in the CTE curriculum to a more explicit level. To reduce 
variation in implementation across sites, facilitators were provided with a trainer manual that 
included sample agendas, lesson plan templates, and other relevant training materials. During the 
fall term, the agendas were refined, and all facilitators attended the first workshop (at Site E) to 
further standardize the workshops to be conducted at the other sites. 

All teachers in the workshops received a packet of materials that contained information about 
the study and their roles and responsibilities as participants. They were also provided with 
curriculum maps aligning math concepts (e.g., algebra, geometry, trigonometry) with existing 
high school CTE curricula for their specific SLMP, and examples of contextualized math lessons 
gathered from various sources around the country, including PISA and various teacher resource 
Web sites. 

At the workshops, CTE-math teacher teams worked together to further revise the curriculum 
maps, using curricula from their schools, districts, and states. These revised maps were then used 
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to identify and select the math concepts around which they would develop math-enhanced CTE 
lessons. Teacher teams at each of the replication sites were asked to agree upon, develop, and 
teach a total of 5 to 10 lessons for their SLMP. 

Eacilitators at each of the workshops introduced the pedagogical framework — the seven 
elements — to the teacher teams (see Eigure 3). Eacilitators also provided teachers with a lesson 
template and two sample lessons that incorporated the seven elements. The teacher teams were 
instructed to include all seven elements in their constructed lesson plans. At the conclusion of the 
workshops, they received copies of the lessons they and their colleagues created, and were 
expected to continue working together to develop and refine the lessons for implementation. 

During the implementation phase in the spring semester, all teachers were expected to teach 
all the agreed-upon lessons created for their SLMP. The teacher partners were asked to 
communicate before and after the CTE teacher taught each math-enhanced lesson. Ahead of each 
lesson, the math teacher provided support for the CTE teacher in planning the instruction, 
answering questions, helping solve problems, and offering encouragement. Then, following the 
lesson, the math teacher followed a structured debriefing protocol with the CTE teacher, and 
entered reflections onto the NRCCTE Web site or sent a written summary via e-mail, fax, or 
standard mail. This procedure also served as an indicator to NRCCTE that the lesson had taken 
place. 



To ensure that the experimental intervention was consistent across the replication sites 
and that the control teachers did not do anything different from the usual, procedures were 
developed to enhance the fidelity of treatment. These procedures are addressed in Chapter 5. 

Data Collection 

A mixed method approach to data collection was employed throughout the pilot study. More 
specifically, this approach is understood as a “concurrent nested strategy” (Creswell, 2003). This 
type of procedure involves one data collection period in which both quantitative and qualitative 
data are collected — one given priority and the other embedded within, as described here: 

Given less priority, the method (qualitative or quantitative) is embedded, or 
nested, within the predominant method (qualitative or quantitative). This nesting 
may mean that the embedded method addresses a different question than the 
dominant method or seeks information from different levels. . . .The data collected 
from the two methods are mixed during the analysis phase of the project, (p. 218) 

Notably, Miller and Crabtree (2000) give credence to using a mixed method approach in 
evidence-based research design. In reference to randomized medical trials, they assert, “Research 
designs in clinical research inherently require multimethod thinking, or critical multiplism, with 
the particular combinations of data-gathering analysis, and interpretation approaches being 
driven by the research question and the clinical context” (p. 619). They further highlight the 
essential interplay of quantitative and qualitative methods in evidence-based research: 
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The new gold standards . . . need to include qualitative methods along with the 
RCT [randomized clinical trial] .... We propose conceptualizing the multimethod 
RCT as a double- stranded helix of DNA. On one strand are qualitative methods 
addressing issues of context, meaning, power, and complexity, and on the other 
are quantitative methods providing measurement and a focused anchor. The two 
strands are connected by the research questions, (p. 613) 

In this study, the predominant quantitative method involved pre- and posttesting of students 
in classrooms and subsequent analysis of the testing data. The pretest provided evidence 
regarding the equivalence of the groups, and permitted adjustment for any pretest differences that 
remained despite the randomization. Nested between the pre- and posttests was the concurrent 
collection of both quantitative and qualitative data used to document fidelity of the treatment, 
and to gain understandings about the teacher experiences during implementation of the math- 
enhancement model. 

Data collection began in fall 2003 with experimental teacher surveys at the beginning of 
professional development, telephone interviews of randomly selected experimental and control 
teachers, and artifact collection from all teachers. At the start of the spring 2004 semester, parent 
and student consent forms, student surveys,^ and math pretests were administered in both 
experimental and control classrooms. At the end of spring 2004, students and teachers once again 
completed surveys, and selected teachers participated in focus groups. Chapter 5 more fully 
describes the qualitative methods used to measure fidelity of treatment and the results that these 
methods produced. The following section describes the quantitative data collection, the results of 
which are reported in Chapter 4. 

Measuring Academic and Technical Knowledge Achievement 

A total of four different math measures were used to assess mathematical performance. Each 
of the quantitative measures provided information to test this study’s specific hypotheses. The 
mathematics section of the TerraNova CTBS Basic Battery (McGraw-Hill, 1997a) was used as a 
math pretest to establish a baseline level of mathematical performance for the experiment. The 
TerraNova CTBS Basic Battery mathematics section was chosen as a measure of academic 
mathematical abilities due to the multiple areas of mathematical concepts covered by the 
measure. Eurthermore, it is a traditional, nationally normed and reliable cognitive test of math 
skills. Level 21/22 was chosen in order to measure high school junior- and senior- level 
mathematics. Although four equivalent forms of the Level 21/22 exam exist, only two forms are 
in print. Of the two printed forms, only Eorm A was available for use in all states involved in this 
study. (Eorm C, the other printed form, was embargoed/blocked for use in two of the sites since 
it contains items used on their high-stakes standardized achievement exams.) As a result. Level 
21/22 Eorm A of the TerraNova CTBS Basic Battery was selected (McGraw-Hill reports 
reliability coefficients of a = .91 for grade 11, and cc - .92 for grade 12). 



® All student and teacher surveys administered for the purpose of this study contain copyrighted materials that 
cannot be reproduced in this report’s appendixes; however, the surveys are available upon request from NRCCTE. 
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The TerraNova CTBS Basic Battery was administered to ensure comparability between 
experimental and control groups at pretest, and if they were not comparable, to be able to use as 
a control when statistically analyzing group differences on the posttests. All students were read 
standard instructions and given 40 minutes to complete the test. 

Eor the posttest measures, the students in each classroom were randomly assigned to one of 
three groups corresponding to the three postmeasures of math ability: 

1. The TerraNova CTBS Survey (McGraw-Hill, 1997b), a shortened version of the 
TerraNova CTBS Basic Battery test, was used as a traditional measure of mathematical 
ability (a = .83 for grade 11, and a - .85 for grade 12). The posttest was different from 
the pretest version (r = .80). 

2. WorkKeys Applied Mathematics Assessment (ACT, 2004b), a measure of mathematics 
used in the workforce, was used as a measure of mathematics in applied contexts. 
WorkKeys is scored using Level Scores and Scale Scores. WorkKeys was originally 
developed for use in job profiling. The instrument traditionally generates Level Scores, 
ranging from 0 to 5, with higher scores indicating greater skills, for use in selecting and 
promoting of individuals within certain professions. However, WorkKeys has recently 
created a second method of scoring, known as Scale Scores, to be used by those using 
WorkKeys data for other purposes requiring a finer level of precision. Scale Scores, used 
for the purposes of this study, allowed groups to be compared on outcome measurement of 
applied mathematics performance. The scores are a function of the number of items 
correct, with higher Scale Scores indicating a higher level of mathematical performance. 
The Applied Mathematics Scale Score, ranging from 65 to 90, employs unequal intervals 
between abilities, and rounds all values to the nearest integer. 

3. The ACCUPLACER Elementary Algebra Test (The College Board, 1998), a standardized 
college mathematics placement test used by many colleges and universities around the 
country, was used as a measure of potential postsecondary remediation requirements 

(a = .92). 



As stated earlier, one-third of consenting students within each classroom were randomly 
assigned to take each of the three math posttests; this was done to reduce the time needed for 
posttesting, as well as to reduce the burden on students and teachers. 

All students also took the skills or content test for their area (e.g. NOCTI, MarkED, AYES). 
The tests used were created by National Occupational Competency Testing Institute (NOCTI) for 
the health occupations, information technology, agricultural mechanics, and horticulture 
replications. The Automotive Youth Educational Systems (AYES) assisted with the automotive 
technology replication, and the marketing education consortium MarkED provided a test for the 
business and marketing replication. 
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The AYES examination is used by schools with ASE-certified automotive programs and 
active chapters of SkillsUSA in order to establish consistency in learning achievements within 
and across their programs. The exams include items in the areas of engine performance and 
repair, suspension, brakes, and electrical and electronic systems. 

The NOCTI tests covered knowledge of the skills used in each respective technical field. The 
health occupations test is used by the National Consortium on Health Science and Technology 
Education (NCHSTE) as part of their certification program, which is recognized in more than 25 
states. Eligible students who successfully pass this online assessment and meet other 
requirements will obtain either a Certificate of Proficiency or Mastery, issued in partnership 
between NCHSTE and NOCTI. 

The information technology, agricultural mechanics, and horticulture tests were created by 
NOCTI specifically for the purposes of this project, using items in its test bank. The MarkED test 
was also created specifically for this project, using items from MarkED’ s test bank; it included 
items from business, marketing, and accounting. 

Each technical skill or content knowledge test included, but was not limited to, the technical 
skills learned in the specific course included in the experiment. The use of these tests allowed 
determination of any loss of technical skill or content knowledge by the experimental group due 
to class time specifically assigned to mathematics, when compared with the control group. All 
students, experimental and control, took the skills test appropriate to their SLMP. 

Posttesting Protocols 

As mentioned earlier, students in each class were randomly assigned to one of the three 
mathematics tests to minimize the study’s intrusion on classroom time and the burden on 
students and teachers. To accommodate simultaneous administration of three different tests in 
one room, the math posttest directions were partially given in writing on the test itself. All 
students were given 40 minutes for each test. 

By design, the development of math-enhanced lessons was unique to each SLMP and, 
therefore, the number of lessons and the number of concepts taught differed across sites. The 
math concepts contained in the TerraNova tests were mapped by the publisher, McGraw-Hill. 
Math experts on the research project team mapped the concepts contained in the remaining math 
tests and in the lessons. The decision to use standardized tests as dependent variables was based 
on their reliability and validity, but this decision caused some of the tests items to be out of 
alignment with the concepts taught in the math-enhanced lessons. Based on the mapped lessons 
and tests, the percentage of concepts that were taught and tested at each site could be determined, 
as shown in Table 3. 
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Table 3. 

Percentage Alignment of Math Taught vs. Math Tested for Each Site 



Percentage of Math Concepts Taught 


A 


B 


C 


D 


E 


E 


Covered on Math Pretest 


76.2 


66.7 


61.5 


73.1 


NA 


64.0 


Covered on TerraNova Posttest 


52.4 


38.9 


42.3 


50.0 


NA 


40.0 


Covered on ACCUPLACER Posttest 


23.8 


38.9 


19.2 


26.9 


NA 


16.0 


Covered on WorkKeys Posttest 


28.6 


38.9 


26.9 


26.9 


NA 


28.0 


Covered on ANY Posttest 


71.4 


72.2 


61.5 


73.1 


NA 


60.0 



Note. NA = Not Applicable. The number and exact lesson(s) taught varied greatly within Site E, resulting in no 
meaningful way to generalize coding to the site level. 



Summary 

This chapter describes the procedures followed in conducting a pilot study that tested 
whether or not enhancing the teaching of mathematics in an occupational context affects student 
performance on standardized tests of mathematical knowledge and skills — without detracting 
from CTE knowledge and skill acquisition. This study was conducted as an experiment with 
randomized assignment of teachers (but not of students) to the experimental and control 
conditions. The study consisted of six separate replications, each in a different occupational area: 
agricultural mechanics, automotive technology, business and marketing, health occupations, 
horticulture, and information technology. 

Each of the replication sites had the primary responsibility for recruiting teachers within their 
occupational area. CTE teachers who were interested in participating had to identify a math 
teacher who agreed to support their teaching of math throughout the study. A total of 274 
teachers applied to participate and were randomly assigned to the experimental or control 
condition. Upon being informed of their assignments, some withdrew, but 236 (114 experimental 
and 122 control) and their nearly 4,000 students participated in the study during the spring of 
2004. 

Teachers assigned to the experimental group took part in professional development 
workshops in the fall of 2003. At these workshops, the CTE teachers worked with their math- 
teacher partners to identify mathematical concepts in their occupational curricula and to develop 
lesson plans to enhance the teaching of these concepts. In the 2nd half of the 2003-2004 school 
year (spring 2004), the CTE teachers delivered these lessons, which constituted the experimental 
intervention. 

Because random assignment was conducted at the teacher, rather than the student, level, 
pretesting was conducted with the students to determine if the randomized teacher assignments 
had produced groups of students that were equivalent in performance on a standardized test of 
mathematics at the start of the study. The correlation between pre- and posttest scores was over 
.80. Across all six sites aggregated, students in the experimental and control groups were 
equivalent in math performance, but within two of the sites, there were significant differences 
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between experimental and control groups (see Table 4). This pretest score was therefore used as 
a covariate in further analyses (see Chapter 4). 

Three different tests of mathematics, as well as tests of occupational knowledge appropriate 
to the content areas, were used in posttesting. Throughout the study, extensive qualitative data 
were collected: interviews, instructional artifacts, focus groups, and class observations. Mixed- 
method analysis was used to establish the extent to which the experimental intervention was 
delivered, how teachers experienced the use of the seven-element model, and if the intervention 
had an impact on student performance on the standardized tests (see Chapter 5). 
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CHAPTER 4: QUANTITATIVE FINDINGS 

This study was designed to test the hypothesis that enhancing the curriculum with 
mathematics and modifying pedagogy in career and technical education (CTE) specific labor 
market preparation (SLMP) courses could improve the math achievement of CTE students. As a 
direct test of this primary hypothesis, students’ math achievement on one of three tests 
administered directly after the experimental intervention were examined. Because teachers (not 
students) were assigned to either the experimental or control group the analyses were at the 
classroom level. The second hypothesis stated that math achievement would be improved 
without affecting the learning of technical skills. This hypothesis was tested by comparing the 
performance of experimental and control classrooms on measures of technical knowledge 
appropriate to each SLMP. 

Math Achievement at the Classroom Level 

Random assignment of students to the experimental or control group would have ensured that 
observed and unobserved differences in students between groups would be no larger than would 
be expected by chance. Such an approach, however, is extraordinarily difficult to implement 
absent a state or local requirement or unique circumstance (e.g., lottery assignments to magnet 
schools or academies). This compromise — random assignment of teachers — offers the advantage 
of ensuring that any differences in teacher motivation or skill — potentially critical confounding 
variables — were randomly assigned across conditions. To ensure that the random assignment of 
teachers had resulted in the creation of equivalent student groups, all students were pretested on 
their math ability, and classroom averages were compared within each replication. The results 
are shown in Table 4. 

Eor all classrooms combined, the random assignment yielded experimental and control 
groups whose pretest performances were virtually identical (less than 1% difference in the 
number of test items answered correctly). At one of the replication sites, the scores of the two 
groups differed significantly at the .10 level. Because of this preexisting difference, student 
pretest scores were used as a covariate in analyses of the posttest data. 

The cut-off for statistical significance was established at .10 because of small sample size 
and to avoid a Type II error. ^ Different supplementary measures to statistical significance are 



The statistical significance between two groups evaluates the probability of obtaining the sample outcome by 
chance (Fan, 2001); i.e., that a viable explanation for the observed differences between the two samples is not due to 
sampling error or chance, and heavily relies on sample size (Cohen, 1990). But statistical significance does not 
directly reflect the magnitude of an effect — it does not evaluate whether results are important (Thompson, 2002), 
and does not tell the researchers what they want to know (Denis, 2003; Kirk, 1996). In studies such as the one 
reported here, where small sample sizes were used in the pilot stage for both the experimental and control groups, 
there is the danger of not detecting a significant effect (Type II error) if the alpha is set too low. Furthermore, even 
the fact that the results are significant using p < .10 does not tell us the magnitude of the changes introduced as a 
result of implementing a new math pedagogy for CTE students. Thus, the effect size is the index that most closely 
explains the dimension of those changes among the students in the experimental group. Statisticians have provided 
detailed examples about the use of effect size (Fan, 2001; Kotrlik & Williams, 2003; Rosnow & Rosenthal, 1996; 
Van Etten & Taylor, 1998). 
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available in such cases, and are generally called measures of effect magnitude (Kirk, 1996). 
Within this report, when a difference has reached statistical significance at the more liberal alpha 
level, Cohen’s d is also reported, to show that there is a practical significance of the effect 
(Cohen, 1988; Rosnow & Rosenthal, 1996). Reporting the effect size in experimental studies has 
received increasingly wide attention, and has become a common requirement for publication 
(American Psychological Association, 2001; Kotrlik & Williams, 2003). 



Table 4. 

Comparison of Experimental and Control Groups by Replication Site on Pretest Performance on 
TerraNova CTBS Basic Battery 



SLMP Site 


Group 


M 


SD 


t 


P 


n 


Overall 


C 


48.703 


10.306 


.491 


.624 


122 




X 


48.077 


9.192 






114 


A 


c 


57.224 


9.634 


1.539 


.140 


13 




X 


49.655 


12.879 






8 


B 


c 


47.331 


6.573 


.932 


.357 


22 




X 


45.285 


7.931 






22 


C 


c 


46.634 


11.104 


-.416 


.679 


27 




X 


47.789 


9.466 






28 


D 


c 


45.208 


8.695 


.450 


.655 


18 




X 


44.000 


7.847 






20 


E 


c 


54.398 


11.502 


-.262 


.795 


22 




X 


55.246 


8.242 






18 


F 


c 


44.346 


7.768 


-1.750 


.089 


20 




X 


48.597 


7.139 






18 



Note. C = control classroom; X = experimental classroom. Scores are percentages of correct responses. 



The primary question of the study concerned the effect of the experimental intervention on 
math achievement. This question was examined first by considering the aggregate performance 
of the experimental classrooms compared to the control or counterfactual classrooms on the three 
posttest measures of math achievement, using the math pretest as a covariate. Though the effect 
size was small. Table 5 shows that after the semester of treatment, students in the math-enhanced 
CTE classrooms scored significantly higher on one of the tests (the ACCUPLACER college 
placement exam) than students in traditional CTE classrooms did, when controlling for prior 
math achievement. (The number of classrooms in the table vary slightly because completed tests 
were not obtained from a few classrooms.) 
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Table 5. 

Mean Classroom Posttest Scores with the Pretest as Covariate 



Posttest 


Group 


M 


SD 


/ 


P 


d 


n 


TerraNova 


C 


50.59 


14.23 


.351 


.617 




119 




X 


50.38 


13.87 




112 


ACCUPLACER 


C 


40.67 


11.87 


7.334 


.007 


0.20 


117 




X 


42.93 


11.24 


114 


WorkKeys 


c 


74.92 


3.60 


.663 


.416 




118 




X 


75.14 


3.34 




110 



Note. C = control classroom; X = experimental classroom. TerraNova and ACCUPLACER values represent 
percentage of correct responses. WorkKeys values are Scale Scores. Effect size calculated using Cohen’s d (Cohen, 
1988 ). 

The same question was then considered, examining each replication independently, 
beginning with a simple sign test to check for significant differences in the pattern of scores 
between control and experimental classrooms. This was followed by a site-by-site analysis using 
a one-way ANOVA with the math pretest as a covariate. 

Table 6 shows the results on the math posttests for each of the six replications separately. A 
total of 18 comparisons can be made between the experimental and control groups at the six 
replication sites (3 tests x 6 sites). Of these 18 comparisons, the experimental group had a higher 
mean score 14 times. Using the nonparametric sign test, the probability of finding a pattern of 
positive difference 14 out of 18 times is less than .04. 

Experimental classrooms in two of the replication sites (A and C) scored significantly higher 
ip < .07) on ACCUPLACER than did their corresponding control classrooms. Effect sizes were 
medium at .33 and .46, respectively. The experimental classroom at site C scored significantly 
higher (p < .10) on WorkKeys than did their control counterparts, with an effect size of .40. 



National Research Center for Career and Technical Education 



39 







Building Academic Skills in Context: Testing the Value of Enhanced Math Learning in CTE 



Table 6. 

Mean Classroom Posttest Scores with Pretest as Covariate, by Site 



Posttest 


Group 


M 


SD 


/ 


P 


d 


n 


Site A 
















TerraNova 


C 


57.908 


13.263 


0.160 


0.694 


NS 


13 




X 


50.771 


19.822 








8 


ACCUPLACER 


C 


45.238 


10.766 


4.029 


0.060 


0.32 


13 




X 


48.431 


8.861 








8 


WorkKeys 


C 


77.331 


3.048 


0.227 


0.640 


NS 


13 




X 


76.193 


3.591 








8 


Site B 
















TerraNova 


C 


46.102 


11.164 


2.631 


0.112 


NS 


22 




X 


47.852 


11.975 








22 


ACCUPLACER 


C 


37.648 


9.002 


0.575 


0.453 


NS 


22 




X 


37.998 


10.863 








22 


WorkKeys 


C 


74.690 


3.481 


0.199 


0.658 


NS 


22 




X 


74.918 


3.178 








19 


Site C 
















TerraNova 


C 


48.974 


15.750 


0.352 


0.556 


NS 


26 




X 


49.111 


12.301 








27 


ACCUPLACER 


C 


39.900 


12.035 


3.492 


0.067 


0.46 


26 




X 


45.348 


11.680 








28 


WorkKeys 


C 


74.807 


3.205 


2.934 


0.093 


0.40 


25 




X 


76.086 


3.240 








28 


Site D 
















TerraNova 


C 


45.405 


12.518 


0.415 


0.524 


NS 


16 




X 


47.057 


12.682 








19 


ACCUPLACER 


C 


36.957 


7.746 


0.497 


0.486 


NS 


16 




X 


38.285 


10.601 








20 


WorkKeys 


C 


72.546 


2.394 


2.632 


0.115 


NS 


16 




X 


73.835 


2.782 








19 


Site E 
















Terra Nova 


c 


60.011 


13.454 


0.154 


0.697 


NS 


22 




X 


62.022 


13.885 








18 


ACCUPLACER 


c 


47.965 


15.990 


0.516 


0.477 


NS 


21 




X 


46.522 


12.295 








18 


WorkKeys 


c 


77.123 


3.911 


1.288 


0.264 


NS 


22 




X 


76.258 


2.787 








18 


Site E 
















TerraNova 


c 


46.630 


12.380 


0.750 


0.392 


NS 


20 




X 


47.070 


12.029 








18 


ACCUPLACER 


c 


37.158 


9.266 


2.814 


0.103 


NS 


19 




X 


44.320 


8.516 








18 


WorkKeys 


c 


73.228 


2.926 


0.373 


0.546 


NS 


20 




X 


73.689 


3.920 








18 



Note. C = control classroom; X = experimental classroom. TerraNova and ACCUPLACER values represent 
percentage of correct responses. WorkKeys values are scale scores. NS = Empty cells; indicate that Cohen’s d was 
not calculated because the effect was not statistically significant. 
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Technical Skills and Content Knowledge 

Technical skill and/or content knowledge were assessed at posttest only, and because each 
replication site involved a different occupational area, the test for each site was different. The 
hypothesis had stated that the CTE curriculum could be enhanced with mathematics without 
losing CTE technical skill or knowledge development. In five sites, a single test was used to 
measure technical skill/content knowledge. In the sixth site, one of four different exams were 
used, depending on the certification test appropriate to the content of the class. 

The hypothesis was supported using classroom-level data (Table 7), and no significant 
differences between the experimental and control classrooms on technical skill/content 
knowledge were found. These results suggest that the instruction time used for the math- 
enhanced lessons did not cause the experimental students to score lower than the control students 
on tests of technical skills and content knowledge. Similar to this study’s findings on the 
academic tests, students in eight of the nine technical skills and content-knowledge tests scored 
higher than the control group (though the difference was not significant), suggesting that, at best, 
the math enhancements can improve students’ occupational outcomes. 

Table 7. 

Technical Skill Achievement by Site 



Posttest 


Group 


M 


SD 


/ 


P 


n 


Site A CTE Test 1 


C 


64.415 


13.240 


-0.374 


0.713 


13 




X 


66.786 


15.500 






8 



CTE Test 2a 


C 


37.008 


3.178 


-1.019 


0.411 


6 




X 


45.526 


14.308 






3 


CTE Test 2b 


C 


39.613 


4.273 


-0.439 


0.689 


9 




X 


43.665 


18.251 






4 


CTE Test 2c 


C 


41.090 


9.071 


-0.909 


0.440 


3 




X 


54.121 


23.117 






3 


CTE Test 2d 


C 


45.083 


6.788 


-0.540 


0.618 


3 




X 


51.022 


17.793 






3 



SiteC 


CTE Test 3 


C 

X 


63.324 

65.985 


10.627 

6.501 


-0.748 


0.464 


12 

13 


Site D 


CTE Test 4 


C 


43.815 


6.123 


-0.586 


0.562 


16 






X 


45.290 


8.431 






20 


Site E 


CTE Test 5 


C 


38.489 


8.346 


0.622 


0.538 


22 






X 


36.987 


6.177 






17 


Site E 


CTE Test 6 


C 


38.523 


6.858 


-0.148 


0.883 


20 






X 


38.829 


5.752 






18 



Note. C = control classroom; X = experimental classroom. Some sites were not able to administer 
technical tests to all classrooms. The large standard deviations for the experimental group in Site B are 
due to one classroom that scored 50% to 75% higher than the mean of all classes on the four tests. 
Cohen’s d not calculated for non-significant findings. 
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Summary 

Overall, the experimental classes outperformed the control classes on the ACCUPLACER 
exam after completing a semester-long “treatment” of math-enhanced CTE lessons. Across the 
six replications and all three posttests, 14 of the 18 differences showed the experimental classes 
scoring higher — a significant pattern of differences. Within two of the six sites, classrooms in the 
experimental group scored significantly higher on the ACCUPLACER posttest than did 
classrooms in the control group, and one just failed to reach the .10 level. At one site, the 
experimental group also had a higher score than the control group on the WorkKeys measure. 
Significant differences were not found on the TerraNova test. Since the random assignment was 
conducted at the classroom level, it was concluded that the intervention (the teaching of math- 
enhanced lessons) produced this difference in math scores. It should be kept in mind that all 
statistical tests controlled for math pretest scores. 

Einally, the instructional time used to teach the math-enhanced lessons did not cause the 
experimental classrooms to score lower than the control classrooms on tests of technical skills or 
content knowledge. These findings indicate that enhancing math skills of CTE students does not 
interfere with occupational skill or knowledge development. 
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CHAPTER 5: FIDELITY OF TREATMENT 

A number of measures, both quantitative and qualitative, were employed to ensure that the 
experimental treatment occurred and to document any variations in the treatment that may have 
influenced student performance on the math exams. In addition to detecting if the experimental- 
group teachers actually taught the math-enhanced lessons, the pilot study sought to explain the 
extent to which the lessons were implemented and to describe what worked with regard to the 
math-enhancement model. Variations in the professional development sessions and in the 
subsequent selection of math concepts and development of math-enhanced lesson plans across 
the replications were also documented. An additional concern was that control teachers might 
enhance the math in their curricula due to the fact they were in a math study. This potential 
Hawthorne effect was addressed, along with measures of fidelity of treatment. 

Data Collection 

The measures used in the pilot study are presented below in the order that data were 
collected. Open-ended interview and survey questions, as well as the questions used for the focus 
groups, were developed in concert with and approved by the NRCCTE research team. 
Importantly, all questions were aligned to the central hypothesis (Miller & Crabtree, 2000), and 
written to elicit data that would “build explanatory frameworks” (Charmaz, 2000) for the 
quantitative findings of the pilot study. It should be noted that some of the questions asked on the 
teacher surveys were included for the purpose of two graduate students’ dissertations; analyses 
of those questions are not included in this report, as they are not the primary focus of this study. 

Prestudy survey of CTE experimental and control teachers. A prestudy teacher survey was 
conducted just prior to the first professional development session for experimental teachers. The 
written survey contained quantitative and qualitative questions designed to assess teaching self- 
efficacy in general, teaching self-efficacy in math, attitudes about math, current practices related 
to math, attitudes about the research project, hopes for outcomes, and demographic information. 
In addition, teachers were asked to complete a checklist of mathematics concepts in the CTE 
subject area that they anticipated covering during the semester of the study. 

Prestudy interviews of CTE experimental and control teachers. Telephone interviews of 
randomly selected experimental and control CTE teachers were conducted at the outset of the 
study. The purpose of the interviews was to gain a better understanding of the math instruction 
that was occurring prior to and apart from this study. To that end, the teachers were asked to 
describe their efforts at teaching math in their classrooms up to that point. They were also asked 
open-ended questions (following Kvale, 1996), about what motivated them to participate in the 
study, how they would describe their approach to teaching, and what they hoped to accomplish 
by participating in the study (see Appendix D). A total of 26 experimental teachers and 27 
control teachers across the six replication sites were interviewed. The phone interviews were 
conducted by an NRCCTE researcher and a research assistant under supervision, and were 
audiotaped and transcribed for analysis. 
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Instructional artifacts. At the outset of the study, control and experimental CTE teachers 
were asked to submit any math-related instructional materials they had been using in their 
classrooms. The purpose of the collection of these artifacts was to supplement the prestudy 
survey and interview data in determining more objectively what math was being addressed in the 
classrooms prior to the study. Instructional artifacts included the following categories: course 
plans/outlines, curriculum, objectives; texts, reading materials; lesson plans, demonstration 
plans; assignments; tests, evaluations, assessments; and student work examples. These materials 
were collected by site personnel at the first professional development session. The materials were 
also used to contribute to the identification of math concepts and development of the math- 
enhanced lessons during the professional development workshops. Instructional artifacts were 
then cataloged and coded by NRCCTE according to type, implementation level, and math level 
addressed (see Appendix E). 

Postteaching debriefings. Math teachers were asked to conduct debriefing sessions with their 
CTE-teacher partners following each math-enhanced lesson. These sessions were intended to 
help CTE teachers reflect on their teaching experience and to engage math-teacher partners in 
actively supporting the CTE teachers throughout the study. A set of questions aligned to the 
NRCCTE model was provided to the teacher teams to help guide the reflections (see Appendix 
E). Once completed, the math teachers were to submit summaries to their site researchers. As a 
fidelity measure, the debriefing sessions were intended to provide evidence that all math- 
enhanced lessons were taught as planned and that the teacher teams were meeting consistently 
during the term. Despite the best efforts of researchers and site personnel to elicit these 
debriefings, the teachers did not submit them with enough consistency to transform the feedback 
into meaningful numeric data. However, many of the submissions provided rich descriptions of 
the model, and were used to illustrate and support themes developed from the triangulation of 
other data sets. 

Classroom observations of experimental CTE teachers. The classroom observations served 
two purposes: 1) to ensure that teachers were implementing the math-enhanced lessons as 
planned, and 2) to capture descriptive data about the math-enhancement model as it happened in 
the classroom. Each experimental CTE teacher in the study was observed once teaching a math- 
enhanced lesson; a total of 1 12 observations were conducted. 

Observers conducted “focused and selective observations” (Angrosino & de Perez, 2000) 
guided by the criteria aligned to the seven-element math-enhancement process. Because 
observations were specifically focused on the implementation of the seven elements, it was not 
possible to utilize a standardized instrument from another study. Therefore, expert teacher 
educators and researchers were consulted in the development of a criterion-referenced 
observation tool and processes (Castellano, Stringfield, & Stone, 2003; Center for Applied 
Research and Educational Improvement, 2000). Using the observation tool (see Appendix G) 
observers recorded the seven lesson elements just as they were taught, using comment boxes to 
record evidence for each math-enhancement element. Observations were not focused on teaching 
quality or efficacy; however, observers were asked to comment about the lesson context, the 
teaching strategies used, lesson barriers, and other anecdotes not related to quality or efficacy. 
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Classroom observations were conducted throughout the pilot study by trained personnel from 
each replication site. To assure continuity in observations across the sites, observers were 
provided with 6 hours of training by a single trainer who traveled to each site for this purpose. 
They were given instruction in the seven elements of the math-enhanced lessons, and they 
practiced using the observation tool in groups by viewing videotapes of math-enhanced lessons 
until consensus was reached on each criterion on the instrument. To increase accuracy, observers 
were also asked to audiotape or videotape lessons when possible. 

Math-enhanced lessons. Center researchers reviewed each of the math-enhanced lessons 
developed by the sites for content accuracy, with respect to both the seven elements and the 
mathematics, as they were under development. NRCCTE and site researchers provided teacher 
teams with feedback and recommended changes. Once lessons were edited by the teacher teams, 
NRCCTE researchers analyzed final versions for the math concepts embedded within the 
lessons. 

Focus groups with CTE experimental and math teachers. At the conclusion of the spring 
term, focus groups were conducted at four of the six replications sites. Participation in the focus 
groups was voluntary; 26 CTE teachers and 22 math teachers participated in a total of eight focus 
group sessions. CTE and math teacher groups were conducted separately, so that participants 
would feel more at ease to respond candidly about their experiences. Eocus groups were 
conducted by one researcher, with an assistant to record notes. There was one exception in which 
another member of the research team conducted the session in consultation with the primary 
interviewer. Eocus group sessions were audiotaped and transcribed for analysis. 

A “question path” (Krueger, 1998; Krueger & Casey, 2000) was developed by the NRCCTE 
research team to guide the discussions. The question path moved from a general question about 
their overall experience to more specific questions about the implementation of the model (see 
Appendix H). CTE teachers were asked to elaborate on the implementation of the seven-element 
model, to comment on the strengths of the model, and to identify barriers to implementation. 

Both math and CTE teachers were asked for their recommendations for the full-year 
implementation of the project. 

Poststudy surveys of experimental and control teachers. All teachers in the experimental and 
control groups were surveyed at the conclusion of the pilot study. As in the presurvey, the 
postsurvey contained multiple sections containing both quantifiable questions and open-ended 
qualitative questions. Teachers were asked about their teaching techniques and their job 
environment. Experimental teachers were asked to comment on the professional development 
provided by the sites, and to identify barriers to implementation and successful aspects of the 
teaching model. Selected phone interviews with control teachers were conducted to ascertain the 
level of math activity in the control schools (see Appendix I). Their responses were used to 
develop a survey for all control teachers. 
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Data Analysis and Findings 

Our goal in the analysis of the treatment was to provide an accurate or credible description of 
what was studied and to present “conclusions that are believable and trustworthy” (McMillan, 
2000). There are a number of strategies that can be used to assure accuracy. In this pilot study, 
accuracy was accomplished primary through “triangulation” of data from multiple sources; that 
is, no single data source stood alone. As data were analyzed, themes emerged. Triangulation of 
evidence from multiple sources of data was then used to “build a coherent justification for the 
themes” (Creswell, 2003). 

As explained in Chapter 3, a mixed methods approach was employed that required 
appropriate use of both quantitative and qualitative procedures and strategies. Therefore, the 
analyses frequently required “data transformation” — that is, the codification of qualitative data to 
provide a numeric representation and, in some cases, the qualification of quantitative data to 
support thematic development (Creswell, 2003; Caracelli & Greene, 1993). Eor instance, some 
text from the qualitative interviews was coded and articulated in numeric form where appropriate 
to support the findings. Quantitative data from surveys were articulated in simple percentages or 
mean scores to support thematic descriptions. 

The qualitative tradition followed was primarily that of grounded theory, in which one 
explores processes, activities, and events for the purpose of building theory (Creswell, 2003; 
Glaser & Strauss, 1999). This tradition is further explicated by Charmaz (2000): 

Essentially, grounded theory methods consist of systematic inductive guidelines 
for collecting and analyzing data to build middle-level theoretical frameworks that 
explain the collected data. . . . The rigor of grounded theory approaches offers 
qualitative researchers a set of clear guidelines from which to build explanatory 
frameworks that specify relationships among concepts, (p. 509) 

Texts were analyzed to identify the emergent themes. Once themes were identified, texts 
were analyzed again and coded by a primary researcher and an assistant to assure accuracy. 
Quotations from the texts were then selected to illustrate the themes that follow here. 

Preexisting Math-Related Activities 

When the level of math-related activity occurring prior to the study was examined, no 
evidence that teachers engaged in any systematic approach similar to the treatment in this study 
was found. As well, virtually no differences in the response patterns between the control group 
and the experimental group teachers were found. While most of the CTE teachers reported 
addressing the math in their courses in some way, they were not engaged in the identification and 
mapping of math concepts within CTE curriculum and/or subsequent development of math- 
enhanced CTE lessons using anything similar to the NRCCTE seven elements. 

Within the prestudy interviews, teachers were asked to describe their efforts to teach math in 
their courses, and to provide some specific examples. At the beginning of the interviews, many 
indicated that they taught little math. However, as the interviews progressed, their descriptions 
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pointed to a number of ways in which they were, in fact, addressing the math in their classes. 

This finding was supported by data from the presurvey in which teachers were asked to indicate 
the frequency with which they taught math concepts in their classes. Given a scale from not at all 
to a great deal, a majority of the teachers indicated a mid-range of some influence. Responses 
spanned from this mid-range toward teaching math concepts “quite a bit” (see Appendix J). 

We found the most common approach to be a cursory walk-through of the math in lessons, 
projects, or problem-solving scenarios. Only about a third of those interviewed mentioned using 
math formulas or math language. Even fewer used related math examples, and, significantly, 
none gave an example of applied math that was extended to a traditional math problem. As one 
experimental teacher indicated at the conclusion of the pilot: “[This model] forced me to 
reevaluate curriculum from a different perspective. . . .1 always taught the [math] concepts, but I 
never used this vocabulary or this approach.” 

Relatively few of the requested artifacts were submitted, and a number of those did not 
contain any evidence of math instruction. The most commonly submitted artifacts revealed that 
math occurred within a lesson or as a lesson within a CTE unit. The level of math indicated in 
the artifacts was predominantly basic mathematics such as measurement, fractions, and 
percentages. Analysis of the artifacts supported other data in that there was no systematic or 
explicit delivery of math instruction in any of the classrooms at the outset of the study. 

A possible exception to this finding was noted in selected classrooms in Site E, where math 
was frequently integral to the curriculum. A number of the experimental teachers were using a 
textbook in which the math was already integrated into the content. Syllabi and course outlines 
also pointed to active math instruction related to many of the topics of instruction within that 
SLMP. 

Throughout the study, control classroom teachers were asked to conduct “business as usual.” 
Poststudy interviews and survey data indicated that control-classroom teachers effectively 
maintained this stance. Two respondents, each from different SLMPs, indicated that they had 
placed more emphasis on math during the term, and that was at the request of their schools or 
districts. Of more concern, however, was the reported level of school-based initiatives in math 
(see Table 8). Over half the respondents from Site E reported such activities; to a lesser degree, 
increased math activity occurred in Sites A and C. It is possible that this activity may have had 
some impact on the difference in scores between control and experimental students, especially at 
site E. This will be discussed further in Chapter 6. 
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Table 8. 

Math-Related Activities Reported by Control Teachers 



Number of control-group teacher respondents 


Site 


A 


B 


c 


E 


E 


All 


Total 


9 


8 


20 


17 


15 


69 


Indicated their school/tech center was engaged in total 
school improvement activities involving math across the 
curricula 


2 


0 


2 


9 


0 


13 


Indicated their school/tech center had asked them to make 
changes in the course involving more emphasis on math 
after submitting a copy of their curriculum to NRCCTE 
researchers 


0 


0 


0 


1 


0 


1 


Indicated they personally made changes in the course 
involving more emphasis on math after submitting a copy of 
their curriculum to NRCCTE researchers 


0 


0 


1 


0 


1 


2 



Note. Site D declined to survey their control teachers. 



Professional Development 

As described in Chapter 3, professional development for experimental teachers took place at 
each of the replication sites in fall of 2004. Site leaders planned the workshops for their 
respective SLMPs in accordance with the guidelines in a common trainer handbook developed 
by NRCCTE researchers and consultants. Drawing from multiple data sources, variations in the 
duration of these professional development sessions were found, as well as in the number of 
lessons agreed upon and developed by the experimental CTE-math teacher teams at those 
sessions. 

Sites A and E split the professional development into two workshops, approximately a month 
apart, for the purpose of providing teachers time to develop lessons between workshops (see 
Table 4). This was the preferred model; however, sites B, C, D, and E provided one-time 
comprehensive professional development sessions due to constraints on the teachers’ schedules. 
This variation was a direct result of the timing of the pilot study, which commenced in the midst 
of an academic year, thus impacting teacher availability. In all cases, teachers continued 
development of lessons and interaction with site leaders past the professional development 
sessions and into the implementation stages. 
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Table 9. 

Scheduling and Number of Professional Development Days 



Site 


A 


B 


C 


D 


E 


F 


Nov. 7-9, 








Oct. 24-25 




2 days 


Nov. 20-22 


Nov. 20-22 


Nov. 20-22 


1.5 days 


Nov. 13-15 


Dec. 15-16, 


2.5 days 


2 days 


2 days 


Dec. 8 


2.5 days 


1.5 days 








0.5 days 





Note. All dates occurred in 2003. 



Within the professional development sessions, CTE-math teacher teams first identified and 
mapped the math concepts that were already in the CTE curricula. (Selection of the math 
concepts will be discussed in more detail in the following section.) The teams worked separately 
to create a lesson or two around one of the math concepts in their curriculum and shared it with 
the other teachers in their SLMP (see Table 10). Teachers at sites A, B, C, and D agreed to 
further develop and teach the same set of lessons, although a particular sequence to the lessons 
was not required. This was possible because the schools and programs within these SLMPs 
followed the same or similar curricula. At Site E, the content was homogeneous; however, there 
were differing needs for sequencing of that content throughout the year. This resulted in 
development of a larger set of agreed-upon lessons from which teachers could choose to teach. 
The most variation was found at Site E, where individual teacher-teams created and taught their 
own lessons. Teachers at this site taught fewer, but longer, lessons that addressed multiple math 
concepts unique to their own particular curricula. This outcome was a function both of 
addressing an exceptionally broad range of content and courses offered within the SLMP, and of 
the predominant constructivist perspectives held by the teachers in that region. 

Table 10. 

Number of Enhanced Lessons Developed and Taught 





Site 


A 


B 


C 


D 


E 


F 


Possible number of enhanced lessons 
available to teachers 


8 


9 


10 


9 


2 


9 


% of teachers who taught all enhanced 
lessons 


75 


64 


77 


72 


72 


67 



Note. At Site E, a total of 26 lessons were developed; however, each teacher taught an average of 2 
lessons that were adapted to their own curriculum. These lessons addressed multiple math concepts and 
were generally longer — lasting several days. This represents an important variation in the model. At site 
F, a total of 17 lessons were developed. Teachers selected any 9 of the 17 to teach. 
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In the postproject surveys, teachers were asked the extent to which the professional 
development equipped them to implement the lessons. Although there was some variation by 
site, teacher ratings of the professional development were favorable, overall (see Appendix K). 
Teachers were also asked to comment on how they would improve the professional development. 
This data was triangulated with other sources to identify the barriers to implementation 
(addressed later in this chapter) and, subsequently, to make changes to the professional 
development in the full-year study. Specifically, the teachers noted the need to improve the 
existing lessons and find a better fit between the lessons and their curricula. Teachers also noted 
the need for math instruction, and more time to share and practice the lessons with each other. 
They made suggestions to improve the organizational aspects of the sessions, including making 
the goals of the study more explicit and making better use of the time within the sessions. 

SLMP Math Concepts Addressed by the Study 

Using the curriculum maps as a starting point, the teacher teams were asked to recognize, 
pull out, and point out the math in their CTE curricula. These curricula were then analyzed on a 
site-by-site basis by a team of math experts who checked the content and compiled a list of math 
concepts addressed within each SLMP. Within each SLMP, a math concept may have been 
addressed multiple times. Table 1 1 shows a breakdown of the number of times each math 
concept was addressed by site. 

Table 11. 

Map of Math Concepts Addressed by Enhanced Lessons in Each SLMP 



Number of Corresponding CTE Math Lessons Addressing the 
Math Concept in Each Site 



Math Concept 


A 


B 


c 


D 


E 


E 


Number & number relations 


4 


5 


4 


5 


10 


1 


Computation & numerical estimation 


7 


8 


4 


4 


9 


7 


Operation concepts 


0 


2 


0 


0 


0 


0 


Measurement 


2 


8 


5 


4 


5 


14 


Geometry and spatial sense 


0 


0 


1 


1 


0 


5 


Data analysis, statistics & probability 


4 


1 


6 


4 


22 


4 


Patterns, functions, & algebra 


5 


5 


4 


4 


15 


3 


Trigonometry 


0 


1 


0 


1 


0 


4 


Problem solving & reasoning 


2 


0 


1 


0 


1 


3 


Communication 


1 


0 


2 


1 


0 


1 



Note. Site E teachers developed many more lessons than teachers at the other sites; but they each taught 
fewer, longer lessons (as shown in Table 10 above). 



50 



National Research Center for Career and Technical Education 






Building Academic Skills in Context: Testing the Value of Enhanced Math Learning in CTE 



As indicated in Table 11, the number and type of math concepts varied greatly across 
SLMPs, with some SLMPs accenting the lower-level mathematics while others highlighted 
higher- level high school math in specific areas such as statistics. The majority of Site E’s lessons 
addressed computation, measurement, and trigonometry, while Site B’s strongest areas were 
computation and measurement. Although Site E focused on numbers and number relations, 
computation, data analysis, and algebra, no established subset of lessons was agreed to or taught 
by the teachers at that site. Sites C and D appeared to be the most well-rounded, with each 
teaching at least four lessons in each the following areas: number relations, computation, 
measurement, data analysis, and algebra. Data analysis was Site C’s most emphasized area, 
while Site D had no particular focus. Site A followed a pattern similar to those in Sites C and D, 
mirroring their diversity of teaching at least four lessons in three of the four areas: number 
relations, computation, data analysis, and algebra. Site A’s area of interest was algebra (see 
Appendix L). 

Implementation of the Math-Enhanced Lessons 

As part of the implementation of the “treatment” in the CTE classrooms, math teachers were 
asked to meet with their CTE partners before each lesson, and then to debrief the CTE teacher 
within 1 week of their teaching each math-enhanced lesson. Data from this measure proved 
inconclusive because math teachers repeatedly failed to submit the postteaching reflections. 

Eocus group and survey data pointed to three possibilities for lack of reporting: 1) CTE and math 
teacher partners were not meeting regularly to debrief, 2) all lessons were not taught as planned, 
or 3) lessons were taught, but time was not taken to submit reports. Best efforts by site and 
NRCCTE researchers to assist the process and collect these debriefings repeatedly failed to 
gamer a consistent response from the teacher teams. Relying on self-reports, 78 of 109 
CTE-teacher respondents replied that they had the opportunity to teach all of the math-enhanced 
lessons that were developed in their professional development sessions (refer again to Table 4). 
This ratio of non-completers was fairly evenly distributed across the sites. 

Throughout development and implementation of the lessons, there was a growing realization, 
by both the teachers and researchers, that a math-enhanced lesson was not necessarily equal to 
the teaching of one math concept, nor was it equivalent to one class period. Eocus groups 
revealed that a number of teachers in the study held an expectation that each math-enhanced 
lesson should be taught in a single class period. This expectation was not experienced in practice, 
as some teachers found themselves teaching lessons that took several days to complete. Eor 
some, the expectation was linked to concern about losing time for, and emphasis on, CTE 
content. Eor example, a few teachers mentioned wanting to have “1 day a week” dedicated to the 
math-enhanced lesson. Among and between the sites, variations were noted in the length of the 
lessons in terms of the length of a class period, levels and numbers of math concepts addressed in 
the lessons, and the time it took for students of varying abilities to grasp the concepts. In other 
words, a lesson with higher levels of math concepts, such as geometry, took longer to teach than 
one that contained basic mathematics. Likewise, teaching a lesson to predominantly special 
education students took longer than teaching the same lesson to students who participated in 
advanced placement courses. This frequently translated into “a time issue,” which was reported 
in the focus groups and the postpilot teacher surveys as the primary barrier to implementation. It 
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also showed up in teachers’ requests for better developed lesson plans and more instructional 
materials created for varying levels of student math abilities. 

Observation data indicated that, with a few exceptions, teachers in five of the six sites were 
relatively consistent in teaching Elements 1-6 of the seven elements. However, data from the 
112 observations indicated that approximately one third did not provide evidence for any 
instruction using Element 7. Element 7 was particularly important to the pedagogic framework 
because it involved opportunities for students to extend their learning into more traditional math 
problems. The shortage of evidence for teachers’ use of Element 7 may be explained to some 
degree by its nature as a student-directed activity. In many of the lessons, it manifested as a 
homework assignment or in projects. Sometimes the teacher carried Element 7 into the next 
day’s instruction as a bridge to the next lesson or learning activity. Eor example, students were in 
a classroom one day for the math-enhanced lesson related to the Pythagorean theorem, and 
progressed to the laboratory the following day to experience its practical application to a 
construction project. Observers simply could not account for the instruction that may or may not 
have occurred in the days that preceded or followed their observations. 

Another explanation for the relative absence of Element 7 was in the quality of the lesson 
plan and in the preparation of the CTE teachers to teach the math. This was confirmed in the 
focus groups by math teachers who noted that extending the math in this manner is challenging 
even for math teachers. As one CTE teacher admitted, “If I didn’t have a clue and [my students] 
were tired, it went by the wayside.” This outcome was, in part, a function of the uniqueness of 
the SLMP and subsequent variations in professional development sessions. All teachers were 
provided with instruction in the seven elements and were given a standard lesson template. 
However, by design, the teachers were given the freedom to create lesson plans as they emerged, 
unfettered by the template as long as the lessons contained the seven elements. A review of the 
lesson plans showed that many chose not to follow the template. While the majority of teachers 
reported liking the seven-element framework and the lessons they developed, they also noted the 
difficulties they faced teaching the lessons they did not author. Reported as a barrier, many of the 
lesson plans were not explicit enough — especially in the math — to help other teachers when the 
time came to teach the lesson. This finding revealed a faulty assumption on the part of 
researchers and teachers, alike, about the capacity to learn the model and successfully teach the 
lessons without highly developed lesson plans that included teacher notes and explicit math 
instruction in the professional development sessions. This was one reason why, in the full-year 
implementation of the study, more emphasis is being placed on math support throughout the 
year. 

Barriers to Implementation 

Time. Teachers identified the predominant barrier to implementation as that of time — both 
the time needed to develop and teach the lessons, and the timing of the study in terms of the 
school year. Since the pilot study was conducted in 1 spring semester, the time needed to teach 
the lessons was not long enough, and often conflicted with required school testing, competitive 
events, spring fever, graduation, etc. Eor many, the time issue was manifested in a poor fit of the 
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math-enhanced lessons to their curriculum, frequently resulting in stand-alone lessons or a set of 
lessons forced into a short time at the end of the semester. 

Fit to curriculum. The math and CTE partners in each SLMP worked together to identify and 
map the math concepts, and subsequently agreed upon a set of enhanced lessons they would all 
teach. This process did not always afford an optimum “fit” to the teachers’ existing curricula. 
Although the CTE teachers taught similar courses, there were variations across their curricula 
both in content addressed and in the sequence of units and lessons. To accommodate this 
variation, they were allowed to teach the math-enhanced lessons in the order of their choosing. 

At one site, teachers were allowed to select a set of lessons from a larger set. Some teachers 
reported needing to “tweak” the lessons to make them work. As indicated above, this was further 
exacerbated by the limits of 1 semester. Teachers were also concerned about whether the lessons 
would prepare students to meet standards for testing and/or certification in the SLMP. An 
exception occurred at Site E, where teachers agreed upon concepts, and then developed their own 
individualized lessons. These lessons were shared only to the extent that others wished to use 
them. 

Inadequately developed lesson plans. CTE and math teachers both identified inadequately 
developed lesson plans as a barrier to implementation. As mentioned earlier, it was easier for 
teachers to teach the lesson(s) they had authored. It was a much more challenging task to prepare 
for and teach lessons with which they were unfamiliar. Some teachers noted that it took an 
extraordinary amount of time to adapt or rewrite the lessons for their own use. When lessons 
were incomplete, or when there were uncorrected errors in the math examples, the need for math 
support intensified. As one teacher contemplated, “There were a couple of those lessons, because 
the problems were wrong, I tended to [doubt] what I was coming up with. . . and I just needed a 
person, the math teacher, to tell me, yes, that is what the answer was.” Overall, teachers 
expressed the need for more worksheets and activities, and a “red-letter” edition of the lessons 
with notes, answer keys to worksheets and tests, and explicit math instruction. 

Bridging the language; knowing the math. It was clear from multiple sources of data that the 
CTE teachers struggled to bridge CTE to the math as they taught the lessons. The task was made 
more difficult when the math on the lesson plan was incomplete or inaccurate. Eor many, the 
experience highlighted their own inadequacies or lack of preparation in math. A number of 
teachers reported calling on their brightest students to help them out — some openly admitting 
their shortcomings to the students. Others found themselves leaning heavily on their math- 
teacher partners, or wanting more access to that partner. In some cases, the result was the 
perception of the lessons as “add-ons” to the CTE curriculum and/or “stand alone” lessons, 
rather than being truly integrated. By the end of the study, math and CTE teachers, alike, 
recommended more explicit math instruction for each lesson, time to practice the lessons in the 
professional development sessions, and expanded lesson plans with worksheets, teacher notes, 
and answer keys. Some teachers suggested the creation of posters for their classrooms with both 
the math and the CTE terms side-by-side to remind them and their students of the natural 
connections. 
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Students’ resistance to math. Prestudy data revealed teachers’ widely held apprehensions 
about student receptivity to the math-enhanced lessons. Teachers noted a wide range of math 
abilities among their students; the majority of teachers reported having low-ability and special 
education students in their classes. They also pointed to the depth of students’ preexisting fears 
and failures with math. In spite of this level of concern, the majority of the teachers supported 
the need for this study. They pointed out that CTE careers require math competencies, and, 
conversely, that math needs the applications that CTE provides. This teacher’s comment reflects 
the hopes of many: “I would like to get students to lose their math anxiety.” Once 
implementation began, teachers noted some initial resistance by students to the math-enhanced 
lessons. However, they also reported that as the students began to understand more math, their 
engagement increased. It was at this point that the barrier became a success for so many. The 
following quote describes the experiences of many teachers in the study: 

I did have some kids that shut down. ... As we were doing the lessons [one 
student] would go, “I don’t know why I need this.” We went through the lessons 
and I was able to explain things to her. She’d go, “This is easy!... But at first I had 
to get over that wall. . . . And she [the student] came out of there with more 
confidence.” 

Successes of the NRCCTE Model 

The turnaround of students’ perceptions and attitudes is one illustration of the many 
successes of the NRCCTE model. Teachers across the study strongly commended this study as 
“essential,” “important,” “necessary,” and “overdue.” Teachers described their experiences as 
“positive” and “worth the effort.” Eor many, their math apprehensions gave way to excitement, 
as one described it: “The first couple of lessons. I’ve got to tell you. . . [I was a] fish out of 
water!” — but later, “having a blast!” Another commented, “The math-enhancement model 
helped me brush up on many of my math skills. . .1 feel this model made me a better teacher as it 
relates to math.” 

Teachers overwhelmingly supported the model. The survey and feedback on the math- 
enhancement model itself was widely positive and supportive. Teachers offered no substantive 
criticism of the seven-element framework itself — only of its implementation presented as a 
barrier earlier in this chapter. In spite of the problems with underdevelopment of the lessons, 
many of the CTE teachers and their math-teacher partners liked the lessons enough to want to 
continue to refine and teach them in the full-year implementation of the study. 

CTE reinforces math skills. When CTE teachers were asked for their perceptions of the value 
of math-enhanced lessons for their students, they were unanimous in their belief that math can be 
learned outside of a math class. Their most frequent responses were that CTE reinforces math 
skills through real-world applications and that CTE classes provide a critical context for 
understanding math and developing stronger math skills. As one teacher noted: “Math that is 
given in authentic and contextual situations is more meaningful to the learner and is more 
powerful than rote practice.” Another indicated: “Math-enhanced lessons will provide additional 
chances/opportunities for students to use real-life application of principles of math in a situation 
[where] they can see [its] relevance.” The math teachers agreed: “I like these ideas. Now I can 
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explain why these concepts are important. I know how to do them, but I never understood how 
you guys use those in real life.” 

Math strengthens CTE skills. Conversely, teachers emphasized the importance of math to the 
understanding and strengthening of CTE skills, noting that good math skills are essential in the 
world of work. This teacher’s comments reflected this perception: “Regardless of the profession 
that a student selects, math will play an important part in the job performance.” An automotive 
instructor noted: “I have found that students with higher math skills are better at diagnostic 
procedures. Use of math is very important in electrical work.” These responses illustrate the 
subtle, but notable, interplay between the perceived value of math and CTE. Teachers from 
automotive technology, health occupations, and information technology were inclined to 
emphasize the importance of math in strengthening CTE. Agricultural mechanics, horticulture, 
and business and marketing were inclined to emphasize the contributions of CTE to math. One 
teacher provided this mediating response: “I believe that math-enhanced lessons will benefit 
students because (a) math is an important subject [and] (b) learning outside the math classroom 
might be the ticket for some students.” 

Benefits to students. When this study began, the CTE teachers widely expressed the need for 
their students to be encouraged and motivated in math because so many students come to their 
classes with negative preconceived ideas and fear about their math abilities. At the conclusion of 
the pilot study, CTE and math teachers noted students’ progress and growing pride as a result of 
participating in the program. As one shared: “My entire class ‘bought’ into the program. The 
sum of the program was greater than the individual math lessons.” Another shared the “fun” of 
encouraging students in the math: 

. . .the kids would go tell the math teacher what I did and then they’d work out a 
problem in math class which the math teachers really liked, even the ones who 
weren’t involved in the program. They’d [the math teachers] come and ask, ‘What 
in the world are you doing. . .our kids come in and ask about problems I didn’t 
think they would be interested in!”’ 

Some teachers reported that students were enjoying the math-enhanced lessons. “They saw 
some things they don’t see in a math class. The students were willing to work harder and 
understand math more when they could put it to work in the lab — it made math more ‘hands- 
on.’” 



There were encouraging stories of successes for low-ability students by CTE and math 
teachers alike. One teacher commented on increased confidence and engagement of her “lower- 
level kids” as they began to coach each other. Another reported how her students were 
struggling, but less frustrated, than in their math classes because they were beginning to relate 
how the math would fit in their field. One teacher noticed an upturn in the number of passes he 
was writing to release students to attend remedial math sessions. “The one thing I am probably 
most proud of out of this whole project [is] that I have more of my kids volunteering to go for 
remedial math than ever before.” 
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Math teachers ’ respect for CTE. A number of math teachers who were located in the same 
building as their CTE partners noted the increase in engagement of students in their math classes: 
“I think the other teachers are jealous of what we are doing. . . . There are better relations between 
CTE teachers and math teachers .... I have found that the students are more willing to accept 
math and also take the tests. They are becoming more accepting of math.” 

Among the most encouraging outcomes of the pilot study were the favorable attitudes of the 
math-teacher partners toward the CTE teachers and the value of CTE classes — one calling the 
experience “eye-opening.” The math teachers were particularly enthusiastic about learning new 
math applications; many noted that they will use the applications in their own classes once the 
study is completed. “It’s about a ‘real-world’ application.... We [try] in mathematics... to get 
real-world applications, but [CTE] is doing it every day — they’re just right in there.” With that 
has come a growing respect for the CTE teachers, as one noted here: “My partner is a 
‘veteran’ .... He’s forgotten more than I’ll ever know.” 

Summary 

Overall, the experimental and math teachers perceived the study and the math-enhancement 
model as a success. In the words of one teacher: “In my opinion, this is a very worthwhile 
program. This is not the first time I’ve been involved in something like this, but this is the only 
thing that’s ever worked.” 

In the pilot study, professional development sessions were planned and conducted by 
educators and researchers at each of the replication sites. There were variations across the sites 
both in the duration of the professional development sessions and in the number of math- 
enhanced lessons that were subsequently developed. The most notable variations occurred in the 
kinds of math concepts addressed in the lessons respective to SLMP content and curricula. 

A majority of the experimental CTE teachers across the study reported implementing all of 
the lessons they developed. Barriers to the implementation of the lessons included time -related 
issues, lack of fit to the curriculum, inadequately developed lesson plans, need for more math 
knowledge and support, and students’ resistance to math. With two exceptions, the control CTE 
teachers appeared to have been successful in conducting business-as-usual without increasing the 
math in their own classrooms through their own efforts. Control teachers in three sites indicated 
that their school or district had mandated increased levels of math-related activities. 

The successes of the NRCCTE seven-element model were many. While teachers offered 
numerous ways to improve the professional development, their evaluations of the sessions were 
favorable, overall. While students were at first perceived to resist the math-enhanced lessons, by 
the conclusion of the study, teachers reported many ways in which their students benefited from 
the process. Both CTE and math teachers noted the reciprocal importance of math to the 
development of CTE skills, and of CTE as a valuable context for learning math. As a parting 
comment in one focus group, a teacher remarked: “I was so glad to see that the program is 
continuing. . . .it’s a valuable project and I wish they could do it in other [academic] areas.” 
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CHAPTER 6: CONCLUSIONS AND RECOMMENDATIONS 

This chapter summarizes the pilot study: its purpose, conduct, and findings. It also describes 
the changes made in the procedures for the full-year study that arose from the formative 
evaluation data collected in the pilot. 



Summary of Purpose 

By most objective measures. United States high school students do poorly in math. Most 
young people who graduate from high school are not prepared for the math they will encounter 
in college, nor in the workplace. College remediation rates are extraordinarily high in this core 
content area (see Hoyt & Sorensen, 2001; Parsad, Lewis, & Greene, 2003; Rosenbaum, 2002). 
Unfortunately, the students most likely to lag behind academically by the end of high school are 
from minority populations and those disadvantaged by socioeconomic status (SES) or disability 
(NCES, 2004). 

United States youth have been taking more math coursework since the 1980s, but math 
scores on the NAEP have remained stagnant during that time (see Castellano, Stringfield, & 
Stone, 2003). These data suggest that doing more of the same is not an effective strategy for 
improving the math skills of high school students. A new approach is required. 

While we know that most students do poorly in math, we also know that most students are 
enrolled in CTE at some point during their high school experience. The most recent National 
Assessment of Vocational Education (NAVE; Silverberg, et ah, 2004) pointed out that many of 
these youth are heavily invested in CTE while in high school. They found that 97% of 
students — virtually all high school students — take some CTE courses. CTE courses that prepare 
students for employment are more likely offered during the junior and senior years — a point at 
which most students have met their 2- or 3-year high school math requirement and are not taking 
or planning to take additional math. 

We also know that CTE course content is rich in the use of mathematics. The CTE curricula 
in use do not routinely emphasize the embedded mathematics, however, and CTE teachers are 
not trained to do so. Thus, the logic of this study was to take advantage of the heavy investment 
of youth in occupational CTE courses later in their high school careers, the potential the 
curriculum offers for enhancing the math skills of CTE students, and the philosophical 
imperative to ensure that CTE students exit high school with more than the economic benefit 
conferred by such coursework (see Silverberg, et ah, 2004) by adding value through improved 
math ability. 



Summary of Study 

This study was designed to test the hypothesis that enhancing the curriculum and modifying 
the pedagogy within career and technical education (CTE) specific labor market preparation with 
mathematics courses (SLMP) could improve the math achievement of CTE students without 
compromising or reducing their technical skill or occupational knowledge development. 
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Working in concert with math and CTE experts, a professional development process and seven- 
element intervention were developed experimentally tested during the course of 1 semester. 

CTE teachers were recruited to participate in the study and assigned them to either the 
control or the experimental group. The control group was asked to keep teaching their CTE 
classes as they had always taught them. The experimental group, along with math-teacher 
partners of their own selection, were brought to a professional development workshop to learn 
about the model for enhancing CTE courses with more math. Together, the teacher teams in the 
experimental group identified the math that was already in the CTE curriculum and worked on 
creating lessons that would emphasize these math concepts. CTE teachers taught these lessons 
over the course of 1 semester, after their students had been pretested to assure equivalence of 
math skills between experimental and control group students. 

To determine whether the students in experimental classrooms gained in math skill over the 
course of the semester, three standardized, global measures of math achievement were used: 
TerraNova, ACCUPLACER, and WorkKeys. Each provided a different perspective on math 
achievement. TerraNova is a traditional math exam used by many states as the basic skills or exit 
exam. ACCUPLACER is a widely used college placement exam. WorkKeys is an applied math 
examination. This approach was chosen because each of the three tests would be differentially 
sensitive to the math interventions. Students were also tested on their occupational content or 
skill achievement, to ensure that students did not lose skill in this area while gaining in math. 

Overall, students in experimental classes scored significantly higher on the ACCUPLACER 
exam than did students in the control classes at the conclusion of the pilot study. Across the six 
sites, on 14 of the 18 global measures of math achievement (three tests x six sites), students in 
experimental classes scored higher than students in control classes — a significant pattern of 
responses. In site -by-site analyses, experimental students in two sites scored significantly higher 
on the college placement exam; and, in one site, they also scored significantly higher on the 
applied math exam. The variation in math concepts emphasized across SLMPs may explain the 
differing results. Eor instance, the concentration on computation, data analysis, and algebra may 
have led to increased scores on the ACCUPLACER as seen in Sites A and C, and on WorkKeys 
in Site C. 

Einally, there were no differences in technical skill or occupational content knowledge 
between the experimental and control CTE classes. These findings indicate that it is possible to 
improve the math ability of CTE students without compromising the acquisition of technical skill 
or content knowledge. Because of the random assignment design, we are confident that the 
findings are a result of the intervention created for this study. 

Limitations 

However, across the six simultaneous replications, a great deal of variability was found in the 
fidelity of treatment. The sources of this variability were both structural and procedural (i.e., the 
extent to which the treatment was implemented as intended). The structural variability arose 
from the kinds and amount of math identified through curriculum mapping across the six 
simultaneous replications. Post-hoc analyses of the concepts assessed by the math tests found 
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that they varied in their alignment to the concepts addressed in the different study replications. 
This meant that in some of the replications, it is possible that the tests were not sensitive to the 
focus of the math instruction in the CTE classes. 

Several methods were used to measure the procedural sources of variability — i.e., the extent 
to which the experimental teachers followed the intervention model and if the control teachers 
changed their usual approach to teaching their CTE courses. These methods included surveys, 
interviews, and instructional artifacts collection from both groups, as well as classroom 
observations and focus groups with the experimental teachers. The design also sought to 
determine whether there were systematic external influences on the control schools that might 
influence their students’ math achievement scores. 

As discussed in Chapter 5, a great deal of variability was found on both dimensions. 
Inconsistencies were found in the treatment implementation. In some instances, teachers did not 
teach all of the math enhancements or did not teach them as they were intended. This meant that 
some experimental students received the treatment, and some received something else. In a few 
sites, students in control schools were exposed to school- wide math improvement projects that 
may have influenced their math scores. 

Concluding thoughts. Despite these limitations, it does appear possible to enhance CTE 
curriculum with math concepts and to measurably improve the math skills of CTE students. The 
pilot study shows that it is possible to achieve math improvement without affecting the students’ 
acquisition of the important occupational content or technical skills provided by the programs. 

As discussed in the earlier chapter on fidelity of treatment, and also summarized above, 
follow-up interviews and surveys indicated that there was a degree of variability in the number 
of enhancements attempted and the degree to which they were implemented. Also, from post-hoc 
analyses, it was found that none of the three tests were particularly sensitive to the math that was 
the focus of the enhancements. Einally, it should be noted, the results of this study come from 
only a single semester of implementing a math-enhanced CTE curriculum. All of this suggests 
the great potential of this approach for improving the math skills of CTE students that would 
come from a systematic, sustained effort throughout an entire, multiyear CTE program of study. 
As a result of these known sources of variation, the model and procedures for the full-year study 
were modified to increase consistency in the professional development and in the 
implementation of the math-enhanced lessons. 

This pilot study contributes to the research literature on math instruction by demonstrating 
how procedural and contextual approaches to mathematics instruction can be combined to 
maximize both students’ understanding of math concepts and their ability to transfer math skills 
to novel situations. This was accomplished with the NRCCTE model — composed of the 
professional development-teamwork component and the pedagogy of the seven elements, which 
dynamically bridge applied and abstract examples. This method is not new, but the way in which 
it was incorporated into the model is progressive and has potential for future refinement and 
application. Students’ ability to understand and apply math in work and testing situations is the 
outcome sought. 
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Evidence was also found that students traditionally deemed suitable “only” for the vocational 
track (e.g., low achievers and others who have been “left behind”) can learn enough mathematics 
using this enhanced curriculum to outscore their peers on a college placement exam 
(ACCUPLACER). While the design could not compare the students’ scores to national norms, 
future research should do so. If the NRCCTE model can help students — not just CTE students, 
but all students (the majority of whom take at least one CTE course in their junior and senior 
year) — pass college placement exams and not require postsecondary remediation in math, this 
would save a tremendous amount of investment on the part of students, their families, their 
colleges, and the general public. 

Though the pilot study did not find significant differences between experimental and control 
groups on the other two tests (TerraNova and WorkKeys), this may be because of lack of 
refinement of the model and its implementation. It is expected that differences will be detected in 
the analysis of data from the full-year study that is currently underway. In addition, in the full 
year study, potential confounding factors such as students’ concurrent enrollment in math 
courses and the number of math courses previously taken have been measured and will be 
controlled for. This and other revisions to the design are discussed in the following section. 

Revisions for Full- Year Implementation 

NRCCTE and site researchers met in the summer of 2004 to review the formative evaluation 
data collected during the spring semester’s pilot study. This review led to a number of changes in 
the procedures planned for the full-year study in the 2004-2005 academic year. These changes 
affected the NRCCTE seven-element model for math-enhanced lessons, the professional 
development provided to teachers, data collection procedures, and the number of replication 
sites. This section describes the specific modifications that were made in each of these areas. A 
final report of the full-year study will be released at the end of 2005. 

Changes in the NRCCTE Seven-Element Model 

Eigure 5 lists the seven steps of the pedagogic framework used in the pilot study and the 
changes made in that framework in preparation for the full-year study based on the formative 
findings of the pilot study. The primary purpose for these changes was to help teachers firmly 
embed the math into the CTE context. Classroom observations and focus groups revealed that 
many CTE teachers struggled to make the connections between math and CTE. This frequently 
resulted in the math-enhanced CTE lessons being taught as separate math lessons. In the design 
of the full-year study, more emphasis was placed on the CTE content being studied and on the 
math as an inherent component of that content. Teacher teams were asked to include explicit 
“bridging” of CTE and math by using the vocabularies of the CTE application and of traditional 
math. 

Another change to the full-year study required the teacher teams to develop lessons following 
a standard template. During the pilot study, teachers were asked to address all seven elements in 
the model, but were not required to follow a specific format. This led to considerable variability 
both within and across sites — especially with regard to expanding the enhancement to traditional 
math examples and including formal assessment of students. Teachers’ comments in the surveys 
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and focus groups showed that the more explicitly the seven elements were developed in the 
lesson plan, the easier they were to teach by others. Based on their input, a standard two-column 
template was developed (see Appendix M). In the left-hand column, the teachers write the script 
for the lesson with specific directions for presenting each element. In the right-hand column, they 
write teacher notes that include answers to questions posed in the script, detailed steps for 
solving specific math problems, and other suggestions for teaching the content. 

A review process was also established to increase consistency across lesson plans. Three 
members of the research staff reviewed each lesson to ensure the math was correct, the math was 
grounded in CTE content, and the plan addressed all seven elements in the model. A rubric was 
established to guide this review. This rubric was also shared with the teacher teams in their 
professional development workshops so they could also use it to critique lessons developed by 
their colleagues (see Appendix N). 

Professional Development 

A major finding in the pilot study was the need expressed by CTE teachers for more math 
support. Such support had been anticipated in the original planning of the study and was the 
rationale behind requiring CTE teachers to identify a math-teacher partner in their applications. 
While the math teachers provided support for CTE teachers during the pilot phase, surveys and 
focus groups revealed that more support was needed. Eor the full-year study, additional sources 
of support were planned to respond to this need: 

• additional professional development sessions 

• math cluster meetings 

• a Web site for each of the SLMPs 

• monitoring of the teaching of the math-enhanced lessons 

The primary support in the full-year study was provided in three professional development 
sessions. These were scheduled at each site for the summer of 2004, midway through the fall 
2004 semester, and midway through the spring 2005 semester. The summer workshops were 
conducted for 4 full days, and the two mid-semester workshops for 2 full days each. The 
CTE-math teacher teams at these sessions worked together to improve the lessons developed for 
the pilot study and to develop some additional lessons. They also participated in critiquing the 
lessons, practiced teaching the lessons, and provided feedback to other teacher teams. The 
practice lessons from these sessions were videotaped and uploaded to the Web sites, enabling 
teachers to review them nearer to the time of instruction. 
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Figure 5. Pedagogic framework for math-enhanced lessons used in the pilot study and changes 
made in the framework for the full-year study. 



Pilot Study 


Lull- Year Study 


1 . Recognize math within your 
class. 


1. Introduce the CTE lesson. 

— no special emphasis on math; lesson should be seen 
as a CTE lesson 


2. Assess students’ math 
awareness. 


2. Assess students’ math awareness as it relates to the 
CTE lesson. 

— similar to pilot study, with increased attention to 
ways to assess all students ’ awareness 


3. Walk through the “pulled 
out” math example. 


3. Work through the math example embedded in the CTE 
lesson. 

— similar to pilot study 


4. “Enhance” the math in your 
lesson. 


4. Work through related, contextual math-in-CTE 
examples. 

— increased emphasis on bridging embedded examples 
and ways in which the underlying concepts are 
presented in traditional math 


5. Reinforce the enhancement. 


5. Work through traditional math examples. 

— increased emphasis on more traditional math 
examples, using both CTE and traditional math 
vocabularies 


6. Check for understanding. 


6. Students demonstrate their understanding. 

— similar to pilot study, but with more emphasis on 
including both CTE and traditional examples 


7. Expand the enhancement. 


7. Eormal assessment. 

— math concept to be included informal assessments 
conducted for the overall CTE unit studied 
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In the pilot study, there was a good deal of variability in the professional development. To 
increase consistency across replication sites during the full-year study, a universal agenda 
template was developed that specified the length of the sessions and the activities to be covered. 
This was developed at a meeting of all research and site staff, and then used by individual sites to 
plan their separate schedules. These schedules were reviewed to ensure consistency. The teacher 
handbooks that were developed for the pilot study also underwent considerable modification and 
expansion for the full-year study. The added material included detailed lists of expectations of 
the CTE teachers and their math partners, the lesson template and rubric discussed above, and 
forms for planning and reporting the teaching of the math-enhanced lessons. Einally, a member 
of the NRCCTE research staff attended each workshop to serve as a resource to the site staff. 

The second source of math support established for the full-year study was labeled “math 
cluster meetings.” Scheduled between each of the professional development sessions, the math 
cluster meetings brought together small groups of teacher teams to discuss math-specific 
concerns and problems in teaching the lessons, and to review math for upcoming lessons. The 
small groups were formed in geographic areas (clusters) within their states, and meetings were 
held at locations the teachers in each cluster could drive to in no more than an hour. Selected 
math teachers in each of the clusters served as “math captains” who were responsible for 
developing the agenda and leading the meetings. Using pre- and postteaching reports of lessons 
taught (discussed below), site directors provided math captains with a summary of the problems 
the teachers in each cluster had encountered. These could then be worked through at the cluster 
meetings. 

A third source of support was the development of a Web site for each of the SLMPs. These 
Web sites provided easy access to many resources. All of the lesson plans and accompanying 
instructional materials developed by the teachers were posted to these Web sites for access and 
downloading by the teachers at that site. Videotapes of the practice lessons were available if 
teachers wished to review how the lessons had been taught by the individuals who wrote them. 
The Web site also had a component called “Ask the Experts.” This was an asynchronous chat 
room where the teachers could post questions about the lessons or the math covered in them. The 
sites also had links to other Web sites that made math resources available. To eliminate the 
possibility that a control-group teacher might come upon these Web sites, each teacher had a log- 
in ID and password to access a specific SLMP Web site; researchers had access to all Web sites. 

The fourth and final support was scheduling and monitoring the teaching of the lessons the 
teachers had developed. This was essential to increasing the consistency of the experimental 
intervention across sites. All formative findings identified considerable variability across sites in 
the teaching of the math enhancements. It was not possible to eliminate all of this variability, so 
a system was established both to collect data about the implementation and to monitor progress. 

The system currently being used in the full-year study consists of a schedule for planning 
when lessons are to be taught and two reports (one prior to teaching and one after) on each lesson 
delivered. When the system works as planned, CTE teachers meet with their math partners to 
review the math concepts to be covered prior to teaching each lesson. Eollowing this meeting, 
the math partners submit reports to site directors on the topics discussed. After they have taught 
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the lessons, the CTE instructors submit reports on the conduct of the classes. (Copies of the pre- 
and postteaching reports are presented in Appendix O) These reports are summarized by the site 
directors for math captains to use at the cluster meetings and then forwarded to NRCCTE staff 
for additional monitoring. If reports are not submitted as scheduled, the site directors contact the 
teachers to ask for explanations. If the site directors do not forward the reports, NRCCTE staff 
contacts them. This system of “checks and balances” ensures that lessons are being taught in a 
timely fashion and that CTE teachers are given the math support they need in a timely fashion. In 
the 1st semester of the full-year study, not all teachers have been submitting the reports as 
scheduled, but the follow-up procedure appears to be encouraging more timely submission. 

Reduction in Sites 

In the pilot study, there were six SLMPs and six replication sites. Eor the full-year study, 
there are four sites and five SLMPs; one of the SLMPs (information technology) has only three 
experimental teachers, and was therefore combined with the geographically closest site for 
purposes of administrative efficiency. Administrative difficulties encountered with the 
horticulture replication led to a mutual decision to discontinue its participation as a replication 
site. 



Overall Summary 

The 1 -semester pilot study yielded results indicating that explicit math enhancement of the 
CTE curriculum in SLMP courses can improve the math skills of students without impairing 
their acquisition of occupational skills and knowledge. A significant difference was found for 
one of the three math posttests, and on 14 of 18 site-specific comparisons, the students in the 
experimental group scored higher than those in the control group. These results were found 
despite differences in the math concepts taught in the six replications, variations in the 
congruence between these concepts and the measures, and different degrees of implementation 
of the intervention. 

Changes made for the conduct of the full-year experiment included revisions in the 
instructional model, more uniformity built into the professional development workshops across 
sites (including adoption of a template and rubric for use in writing and revising all lessons), and 
increased math support, which includes closer monitoring of the delivery of the intervention. 
These changes are expected to result in a more rigorous test of the hypothesis. As this is written, 
the full-year study is being conducted. Posttesting will be conducted at the end of the academic 
year. The report of the full-year intervention will be available at the end of 2005. 
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APPENDIX A 

SAMPLE CURRICULUM MAP 

The following table is a list of the math identified as part of an agricultural mechanics 
curriculum. The math applications are similar to those that you might include in your Spring 
curriculum. Please use the list as a starting point in your discussions of CTE math enhancement. 
The items you ultimately choose to enhance do not necessarily have to be on the list, but should 
be at least at the algebra and geometry levels, if at all possible. 



Appendix Table Al. 
Sample Curriculum Map 



Agricultural Mechanics Problem- 
Solving Application 


Mathematics Content 
Standard 


PASS 

Standard 


NCTM 

Standard 


Determine sprayer nozzle size, 
given flow rate and speed 


Problem solving involving 
cross-sectional area, volume, 
& related rates 


Process 
Standard 1: 
Problem 
Solving 


Problem-Solving 
Standard for 
Grades 9-12 


Determine pipe size and water flow 
rates for a water pump 


Problem solving involving 
cross-sectional area, volume, 
& related rates 


Determine amount of paint needed 
to paint a given surface (calculate 
surface area, etc.) 


Problem solving involving 
surface area, ratios & 
proportions 


Determine the concrete 

reinforcements and spacings 
needed when building a concrete 
platform or structure 


Problem solving involving 
cross-sectional area, volume, 
& related rates 


Determine measurements in feet 
and inches, as well as metric 
equivalences (meters and 
centimeters) 


Conversions (English-metric 
and/or within each system) 


Algebra I 

Standard 2-8a 


Measurement 
Standard for 
Grades 9-12: 
Apply 
appropriate 
techniques, 
tools, & 
formulas to 
determine 
measurements 


Determine torque wrench 

conversions (foot pounds, etc.) 


Conversions (English-metric 
and/or within each system) 


Determine temperature conversions 
(Fahrenheit and Celsius) 


Conversions (English-metric 
and/or within each system) 


Develop different bale-stacking 
schemes that maintain balanced 
loads on a trailer bed of a given 
dimension 


Problem solving involving 
volume & weight 


Geometry 
Standard 2-4 


Measurement 
Standard for 
Grades 9-12 


Determine the time needed to cut a 
field of a given acreage 


Problem solving involving area 
& related rates 


Determine the volume of a fuel tank 


Calculating volume 


Determine engine displacement 


Calculating distances is three- 
dimensional space 
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Agricultural Mechanics Problem- 
Solving Application 


Mathematics Content 
Standard 


PASS 

Standard 


NCTM 

Standard 


Calculate the dimensions of a gate, 
panel, loading ramp, or chute, 
and the number of board feet 
required to build it 


Calculating surface area & 
estimating materials 


Geometry 
Standard 4-4 


Geometry 
Standard for 
Grades 9-12 


Calculate lengths of diagonals using 
the Pythagorean theorem for 
designing and building gates, 
panels, ramps, chutes, etc. 


Solving problems using the 
Pythagorean theorem 






Calculate the bill of materials, 
accounting for waste, efficiency, 
etc. 


Estimating costs 






Calculate and use scales for 3-D 
drawing 


Calculating & using scales 
(ratio and proportion) 


Geometry 
Standard 2-2 
and 2-5 


Geometry 
Standard for 
Grades 9-12 


Determine the amounts of sand, 
aggregate, concrete mix, water, 
etc., needed to make a given 
amount of concrete 


Solving mixture problems using 
ratio & proportions 






Calculate the required dimensions 
of a bunker or tank to hold a 
given volume of feed or fuel and 
one of the cylinder’s dimensions 


Calculating cylinder dimensions 
given volume & one of the 
dimensions 


Algebra I 
Standard 1-1 
and 6a 


Algebra Standard 
for Grades 
9-12 


Design bale feeders with equal 
sections 


Using ratio & proportion to 
solve problems 






Build a materials list for a given 
project (examples: lbs of penny 
nails, number of 2x4s, number of 
2x6s, etc.) 


Calculating materials using 
estimation, ratio & 
proportion, charts, & graphs 






Determine center/midpoint of a 
board or area when calculating 
center of gravity, etc. 


Calculating center/midpoint of 
a line or area 






Use appropriate graphs and charts 
to determine welding rod 
thickness to voltage (and/or 
amperage) to metal thickness 
relationships 


Using composite graphs to 
solve problems 


Algebra 1 
Standard 3- la 
and 3- lb 


Data Analysis & 
Probability 
Standard 


Read and interpret values from tap 
and die charts when drilling on 
metal 


Reading & interpreting graphs 






Read and interpret safety charts to 
determine exposure limits for a 
potentially unsafe element 
(example: excessive noise) 


Reading & interpreting graphs 
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Agricultural Mechanics Problem- 
Solving Application 


Mathematics Content 
Standard 


PASS 

Standard 


NCTM 

Standard 


Use tables and graphs to determine 
compression ratios 


Reading and interpreting graphs 


Algebra 

Standard 2-8b 


Problem Solving 
Standard 


Calculate the amount of 

compression/pressure to use for a 
given set of project specs 


Solving problems involving ratio 
and proportions 






Use histograms and scatter plots of 
safety data in making decisions 


Reading and interpreting graphs 


Algebra I 

Standard 2-5b 
and 3-2 


Data Analysis & 
Probability 
Standard 


Determine flow and distribution 
rates for a given nozzle 


Reading and interpreting graphs 






Graph and interpret time spent and 
cost of projects 


Reading and interpreting graphs 






Chart and interpret water flow and 
restriction for a given pump 


Reading and interpreting graphs 






Plot distribution of seeds from a 
seed drill, and use to determine 
equal distribution (uniformity) 


Reading and interpreting graphs 






Chart water flow differences 

through straight or bent pipes and 
pipes of different sizes; use the 
charts to determine the best pipe 
for a given water flow 


Reading and interpreting graphs 
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APPENDIX C 

ANALYSES OF ATTRITION AND EQUIVALENCY OF GROUPS 

Field studies inevitably incur some attrition from the sample initially selected for 
participation. This appendix examines the effects of teacher attrition on the characteristics of the 
schools that participated in the study. Also examined here: if the random assignment, which was 
conducted at the teacher level, resulted in experimental and control groups of students with 
similar characteristics. 



Teacher Attrition 

Data were assembled and analyzed to determine if the teachers who withdrew after being 
assigned to the experimental and control groups differed from those who participated. Limited 
individual data about the teachers was available from their applications, including the names of 
the high schools and career centers in which they taught. Using this information, data was 
assembled on some of these schools from the Common Core of Data compiled by the National 
Center for Education Statistics, U.S. Department of Education, to compare the characteristics of 
these schools. 

Data was accessed on 87 schools and career centers, all but 12 of which were comprehensive 
high schools. The Common Core of Data either did not provide or provided incomplete data for 
most career centers. This is because most students in career centers attend on a half-day or other 
shared-time basis, and they are counted as part of the enrollments of their sending high schools, 
not of the centers themselves. 

From the total 87 on which data was obtained, 69 were schools and centers in which teachers 
administered the experimental intervention, and 18 were schools and centers where teachers 
applied but withdrew before participating. These two sets of institutions were compared in terms 
of total enrollment, percentage of non-Hispanic European/ Anglo enrollment, and percentage 
eligible for free or reduced-price lunches. The results are shown in Appendix Table Cl. For each 
of these comparisons, there were no significant differences between the schools of the 
participating teachers and the schools of those who withdrew. 

These comparisons yield no information about the teachers, themselves, but they do imply 
that the characteristics of students of the participating and withdrawing teachers were similar. 
Separate analyses were done within the specific SLMPs if there were at least three schools with 
teachers who both participated and withdrew. Four such comparisons were possible, and no 
significant differences were found. This analysis and the relatively low overall percentage of 
withdrawal (13.5%) suggest that teacher attrition did not have significant impact on the study’s 
results. 
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Appendix Table Cl. 

Comparison of School Enrollment Data for Experimental Teachers Who Participated in the 
Study and Those Assigned Who Withdrew 



Enrollment Data 


Participated 


Withdrew 


t 


P 


M total enrollment 


986.42 


1084.676 


.49 


.31 


SD 


710.04 


767.18 






M % of non-Hispanic 
European/ Anglo enrollment 


76.38 


75.61 


.12 


.45 


SD 


22.00 


22.99 






M % eligible for free or 
reduced-price lunch 


28.50 


25.86 


.53 


.30 


SD 


19.41 


18.23 






Base number for M 


69 


18 







Equivalency of Experimental and Control Groups 

To determine if random assignment of teachers resulted in students with similar 
characteristics in the experimental and control groups, a number of individual characteristics of 
students were compared. Two of these — gender and one category of race/ethnicity — were 
reported in Chapter 3. Appendix Table C2 presented the results for five categories of 
race/ethnicity. These additional categories do not change the findings reported in Chapter 3: the 
only significant difference occurred in Site A, which had a high percentage of students of 
African-American decent in the experimental group and a corresponding lower percentage of 
non-Hispanic European/ Anglo. 

Appendix C also reports the analyses of six additional characteristics that preliminary 
regression analyses found to be associated with outcome measures in at least two of the SLMPs. 
The characteristics on which the groups were compared were the following: 

• number of math courses taken, as self-reported on the prequestionnaire 

• Grade Point Average (GPA), as self-reported on the prequestionnaire 

• total math anxiety score; the sum of six 5-point self-ratings (minimum score 6, maximum 
30), with higher scores indicating lower levels of anxiety 

• how far the student plans to go in school, as reported on the prequestionnaire; level was 
coded on a scale of 1 to 8, and means were calculated on the coded values 

• number of other CTE courses taken, as self-reported on the prequestionnaire 

• hours the student studies daily for the course in which the math enhancements were 
taught, as self-reported on the prequestionnaire 

The results of these comparisons are shown in Appendix Table C2. 
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Appendix Table C2. 

Racial/Ethnic Characteristics of Experimental and Control-Group Participants by SLMP 



SLMP 


Non-Hispanic 

European/Anglo 


African- 

American 


Hispanic 


Asian 


American 

Indian/Pacific 


Base 

Numbers 


Site 


X 


C 


X 


C 


X 


C 


X 


C 


X 


C 


X 


C 


A 


44.4 


78.7 


46.8 


7.1 


3.2 


3.2 


2.4 


0.0 


3.2 


11.0 


124 


155 


B 


76.0 


73.9 


4.4 


8.1 


10.7 


7.0 


1.3 


1.7 


7.6 


9.2 


317 


357 


C 


61.6 


60.6 


24.0 


27.3 


5.1 


5.0 


2.0 


1.2 


5.9 


6.4 


375 


421 


D 


72.8 


65.8 


3.1 


3.7 


8.2 


13.6 


2.8 


3.4 


13.1 


13.6 


389 


295 


E 


62.3 


66.7 


6.2 


6.4 


18.1 


17.3 


5.9 


4.4 


7.4 


5.3 


337 


342 


F 


62.6 


65.1 


3.9 


2.6 


4.5 


4.7 


1.1 


0.0 


27.9 


27.6 


178 


232 


Total 


65.8 


67.4 


11.7 


10.8 


9.2 


8.9 


3.0 


2.0 


10.3 


10.9 


1721 


1802 



Note. X = experimental group; C = control group. Statistics are based on 3,521 participants who reported their 
racial/ethnic characteristics. The distributions of these characteristics within SLMPs differ significantly between the 
experimental and control groups only for Site A (X^= 65.90, p < .001). 



Appendix Table C3. 

Comparisons of Experimental and Control Group Students, by SLMP, on Characteristics Pound 
to Be Significantly Associated with Outcomes in Preliminary Regression Analyses 



SLMP 

Site 


Math Courses 
Taken 


GPA 


Math Anxiety 
Score 


Base Number 
for M 


X 


C 


X 


C 


X 


C 


X 


C 


AM 


2.99 


2.95 


2.89 


2.88 


18.57 


19.00 


131-136 


158-168 


SD 


1.08 


1.14 


.58 


.69 


4.14 


4.14 


BM 


2.79 


2.73 


2.64 


2.69 


18.01 


18.03 


216-351 


339-396 


SD 


1.17 


1.00 


.85 


.72 


4.48 


4.26 


CM 


3. OP 


5.27“ 


3.05 


3.04 


17.30 


17.56 


396-434 


433-451 


SD 


1.07 


1.09 


.62 


.59 


4.58 


4.27 


DM 


2.47 


2.32 


2.95“ 


2.7S“ 


17.51 


17.42 


383-424 


280-314 


SD 


1.41 


1.08 


.69 


.80 


4.59 


4.62 


EM 


3.03 


2.91 


3.03 


3.12 


18.61 


18.68 


305-361 


316-359 


SD 


1.39 


1.40 


.66 


.65 


3.91 


3.98 


EM 


2.63 


2.70 


3.11 


3.09 


17.35 


17.31 


193-205 


210-242 


SD 


1.27 


1.11 


.68 


.60 


4.61 


4.10 


Total, M 


2.81 


2.84 


2.96 


2.94 


17.82 


17.94 


1,624-1,904 


1,736-1,924 


SD 


1.27 


1.18 


.70 


.70 


4.44 


4.27 



Note. X = experimental group; C = control group. Base numbers vary because of missing data. The lower number in 
all groups is for GPA. The «s for math courses and math anxiety differ by a maximum of 5 from the largest n 
reported for each group. “The Ms shown in italics differ significantly between the experimental and control groups 
at the .05 level or less. 
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The mean values for the experimental and control groups on these six variables were 
compared in each of the six SLMPs and for all SLMPs combined, yielding a total of 42 
comparisons. Of these 42, five significant differences (p = .05 or less) were found in individual 
SLMPs, and one for the combined SLMPs. The means that differed significantly are shown in 
italics in Appendix Table C3. Pour of the six significant differences were found for the variable 
“number of other CTE courses taken.” On this variable, the total experimental and control groups 
were different, as were three of the SLMPs. Even on this variable, however, the pattern of 
differences is not consistent. Of the three statistically significant differences, the experimental 
group is higher in two. Across all six SLMPs, the experimental group is higher in four. 

The random assignment did not yield uniform comparability between the experimental and 
control groups on all variables, but these tests indicate the groups are far more similar than they 
are different. It is reasonable to assume that random assignment resulted in groups in which 
measured and unmeasured factors that may have influenced outcomes were for the most part 
equally distributed. 

Appendix Table C3, Continued. 

Comparisons of Experimental and Control Group Students, by SLMP, on Characteristics Found 
to Be Significantly Associated with Outcomes in Preliminary Regression Analyses 



SLMP 

Site 


How Far Student 
Plans to Go in SchooL 


Number of Other 
CTE Courses 


Hours of Daily 
Study for Course 


Base Number 
for M 


X 


C 


X 


C 


X 


C 


X 


C 


AM 


4.98 


4.90 


2.74“ 


7.43“ 


1.49 


1.65 


129-135 


163-168 


SD 


1.50 


1.35 


2.28 


1.51 


2.40 


2.06 


BM 


3.79 


3.57 


7.94“ 


7.25“ 


2.71 


2.46 


310-344 


357-390 


SD 


1.54 


1.39 


2.42 


1.45 


3.25 


2.94 


CM 


5.49 


5.60 


.62“ 


.95“ 


2.79 


2.87 


416-433 


435-447 


SD 


1.46 


1.46 


.99 


1.28 


2.82 


2.67 


DM 


4.54 


4.37 


2.99 


2.65 


1.37 


1.66 


404-413 


293-306 


SD 


1.76 


1.89 


2.87 


2.53 


1.88 


2.21 


EM 


5.26 


5.34 


1.85 


1.78 


1.32 


1.21 


345-357 


345-361 


SD 


1.54 


1.52 


1.90 


1.74 


1.54 


1.84 


FM 


4.38 


4.44 


2.15 


2.48 


1.32 


1.74 


194-202 


232-241 


SD 


1.68 


1.74 


2.32 


2.02 


2.28 


2.39 


Total M 


4.78 


4.75 


7.SS“ 


7.6S“ 


1.93 


2.03 


1,798-1,880 


1,825-1,913 


SD 


1.70 


1.72 


2.32 


1.87 


2.51 


2.54 



Note. X = experimental group; C = control group. Base numbers vary because of missing data. The lower number in 
all groups is for GPA. The «s for math courses and math anxiety differ by a maximum of 5 from the largest n 
reported for each group. ‘'The Ms shown in italics differ significantly between the experimental and control group at 
the .05 level or less '’Means calculated using the following coded values; 

1 = Less than a high school diploma 5 = Bachelor’s degree 

2 = Diploma or graduation equivalent (GED) 6 = Master’s degree 

3 = Vocational certificate 7 = Professional degree (MBA, JD) 

4 = Associate degree (AA) 8 = Doctorate 
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APPENDIX D 

PREPROJECT TEACHER INTERVIEW QUESTIONS FOR PHONE INTERVIEWS 
WITH RANDOMLY SELECTED EXPERIMENTAL AND CONTROL CTE TEACHERS 

1. What has motivated you to participate in the math-in-CTE project? [Request a general 
overview of the course(s) that is/are being used for the study.] 

2. In general, how would you describe your approach to teaching? to teaching CTE? 

3. How would you describe your efforts to teach math in your CTE courses? 

a. Are there some examples you are willing to share? 

b. How would you describe your students’ perspectives/attitudes about math? 

4. What do you hope to gain or accomplish by participating in this study? 
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APPENDIX E 

COMPARISON OF THE VOLUME OF MATH INDICATED 
IN PREPROJECT ARTIFACTS 



SLMP 

Site 


No. of 

teachers who 
submitted 
artifacts, per 
site 


No. of 
artifacts 
submitted, 
per site 


% whole 
curriculum 
organized 
around 
math 


% math- 
oriented 
units 


% math- 
oriented 
lessons 
within 
units 


% 

mention 
of math in 
lessons 


% no 
math 


A 


1 


1 






100 






B 


12 


15 


7 




53 


33 


7 


C 


29 


83 


1 


6 


19 


37 


33 


D 


3 


6 






17 


33 


50 


E 


19 


43 


12 


19 


35 


21 


16 


F 


23 


49 




2 


51 


35 


12 


TOTAL 


87 


197 


4 


7 


34 


32 


22 
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APPENDIX F 

POSTTEACHING DEBRIEFINGS 
Debriefing Questions 

1. In general, how did it go? 

2. How did your students respond? In your opinion, did students understand the math 
concepts? 

3. What elements of the enhancement were particularly effective? 

4. What would you like to build on or strengthen? 

5. What elements of the lesson were challenging or difficult to teach? 

6. Were there some elements of the lesson you did not have an opportunity to teach? 

7. If so, why were you unable to teach some elements of the lesson? (If the teacher answers 
lack of time, please identify what caused the time crunch.) 

8. What would you like to do differently next time? 

9. What kind of support do you need to prepare for the next enhancement? 



Appendix Table El. 

Number of Responses by SLMP 



SLMP 

Site 


Total no. of debriefs 
on file 


No. of teachers who 
submitted debriefs 


Average no. of 
debriefs per teacher 


A 


51 


8 


6.4 


B 


138 


21 


6.6 


C 


218 


25 


8.7 


D 


104 


14 


7.4 


E 


40 


17 


2.4 


F 


118 


15 


7.9 


TOTAL 


669 


100 


6.7 
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APPENDIX G 

CLASSROOM OBSERVATION INSTRUMENT* 

Date: 

Number of students in classroom: M F 



Title/topic of math-enhanced 
lesson: 



IMPORTANT: 

• Please attach the lesson plan you were given in advance of the observation. (You should review and 
code the lesson plan before your observation.) 

• Please attach any additional instructional materials you collect. Examples: revised lesson plan, student 
worksheets, written homework assignments, PowerPoint notes, etc. 

• Submit the observation form with attached materials to your site researcher. Sites will send copies to 
the NRCCTE. 



Please make general comments here: 



* The observation coding that appears in this instrument is derived from Castellano et al., 2003, and Center for 
Applied Research and Educational Improvement, 2000. 
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Math-Enhancement Codes; 

These codes do not presume a step-by-step presentation of the lesson. A teacher may choose 
to order the lesson as he or she wishes. 

1. teacher recognizes math with the class (“pulls and points out math”, “talks out loud 
about math” 

2. teacher assesses students’ math awareness 

3. teacher walks through the “pulled out” example 

4. teacher explains math concept(s)/principle(s), integrating math language with CTE 
language (lesson “enhancement”) 

5. teacher reinforces by having students try a similar CTE and math examples 

6. teacher checks for understanding; students demonstrate understanding 

7. students create new CTE and math examples 



Type of Instruction Codes; 

These codes will help us learn more about how the enhanced lesson was delivered. These 
may be added by the observer sometime after the lesson is completed. More than one code 
can be used to describe an activity. 



A 


assessment of student learning 


PM 


teacher problem-modeling 


CD 


class discussion 


Q 


teacher questioning 


CL 


cooperative learning activity 


R 


review of assignments/tests/projects SD 


HO 


hands-on; experiential activity 




student-led discussion/activity 


HW 


assigned homework 


SG 


small-group discussion/activity 


IN 


independent student work 


T 


use of texts, reading materials 


L 


lecture 


TD 


teacher demonstration 


LA 


laboratory activity 


TIS 


teacher interacting w/individual students 


ED 


lecture with discussion 


WW 


worksheet work/writing 


0 

OC 


other (please describe) 
out-of-classroom 

(field exp., shop, greenhouse, etc.) 


UT 


use of computer, calculators, technology 
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Record your observations in 5-minute intervals. Note: More than one math-enhancement code may be used in 
each box. 

Min. Math- Script of Lesson (script what was Method (indicate how the lesson was Type of 

Enhancement taught) taught; note context/location of lesson; Instruct. 

Code Indicate Start Time; describe artifacts that cannot he Code 



collected) 
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APPENDIX H 

POSTPROJECT TEACHER INTERVIEW AND FOCUS GROUP QUESTIONS 
Postproject Experimental CTE Teacher Interview and Focus Group Questions 

Oral Consent and Introductions 

As you think back over this year, what has it been like to participate in this study? 

What was it like to teach math-enhanced lessons? Tell us about it. 

With the seven-element model in mind: 

What are the strengths of the model? 

What specific steps or aspects of the model worked best? 

What kinds of barriers did you encounter in planning and teaching the lessons? 

What kind of math-related challenges did you encounter? 

How did your students respond the lessons? (Ask for stories.) 

How would you improve or change the model? 

If you could design the professional development for the full-year implementation of the study, 
what would you be sure to include? 

What kinds of support or training do you need to plan and prepare for the full-year 
implementation? 

What was it like to work with a math-teacher partner? 

What would you recommend to strengthen the CTE-math teacher partnerships? 

Think back on this year. What have you learned or gained from the experience so far that will 
impact your teaching of math in the future? 

Do you have any final comments or recommendations? 
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Postproject Math Teacher Interview and Focus Group Questions 

Oral Consent and Introductions 

Tell us about your experience participating in the pilot study. 

What can you tell us that will help us plan for full-year implementation of the study? 

Erom your perspective, what worked? What didn’t? 

What do we need to do to make it better next year? 

If you could design the professional development for the full-year implementation of the study, 
what would you be sure to include? 

If needed: 

What do you need to strengthen your role in the project? 

What kind support do you think your CTE teacher needs during the full-year 
implementation? 

Erom a math perspective, what can you tell us about the math-enhancement model (the seven- 
step approach)? 

Strengths? Weaknesses? 

If needed: 

What kinds of barriers did your CTE partner encounter in planning for and 
teaching the lessons? 

What specific steps or aspects of the model seemed to work best for your CTE 
teacher partner(s)? 

What would you recommend to improve or change the seven-step model? Specifically, 
what can you tell us about Step 7, the abstraction (extension)? 

What kinds of math-related challenges did you assist your CTE-teacher partner with? 

If not already mentioned, ask about the debriefings. 

Think back on this year. What have you gained or how have you benefited from being a part of 
the project? 

So you have any final comments or recommendations? 

Summary of Notes 
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APPENDIX I 

POSTSTUDY CONTROL CTE-TEACHER DEBRIEFING INTERVIEW QUESTIONS 

Thank you again for participating as a control teacher in the Math-in-CTE study this past 
year and for making time for this interview. As you know, we have some exciting news, and that 
is we are continuing the Math-in-CTE study for another year. [If you are continuing with the 
full-year implementation of the study, your role will remain the same as last year — to conduct 
business as usual.] 

We have some brief questions for you. Before we start, I want to make sure that you 
understand that the interview will be recorded and transcribed. Your name will be changed to an 
ID number and will not be associated with any data from this interview. Transcriptions are kept 
in a secure area. In any reports we publish, we will not include any information that will make it 
possible to identify you, your students, or your school. 

Do you have any questions before we begin the interview? 

How did last year go? 

• What kinds of initiatives or changes has your school been involved in this past year? 

o How have those initiatives or changes impacted you or your students? 

What kinds of professional development did you participate in this past year? 

• Was any of the professional development math-related? 

o In what ways? Can you describe it for us? 

Were you asked by your school or district to make any changes to the course used for this study? 

• What kinds of changes were requested [for last year and/or the coming year]? 

• Are any of the changes in some way related to math? 

o Would you describe those changes for us? 

Did you [and/or will you] make any kinds of changes to the course used for the study? 

• Did you [will you] change anything related to math? 

o Would you describe those changes for us? 

Do you have any comments or questions? 

Once again, thank you for your time and your commitment to the study. 

[Eor noncontinuing teachers] We appreciate your contributions to our pilot study in Year 1. 

[Eor continuing teachers only] And, thank you for conducting “business as usual” for one more 
year. If you haven’t already been contacted, your liaison will contact you to schedule the student 
consent, survey, and testing dates. If you have any questions, please contact: [site coordinator, e- 
mail, and/or phone]. We will contact you for an interview at the end of next year. Your 
professional development will be scheduled for the summer of 2005. 
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APPENDIX J 

FREQUENCY OF MATH CONCEPTS ADDRESSED IN CTE COURSES 
AS REPORTED BY EXPERIMENTAL AND CONTROL TEACHERS IN 

THE PRESTUDY SURVEY 



Appendix Table Jl. 

Frequency of Math Concepts Appearing in CTE-Teacher Classes 

On a scale of 1 to 9, where 1 is not at all and 9 is a great deal, how frequently do you teach math 
concepts in your CTE classes? 



1 = Not at all, 3 = Very little', 5 = Some influence', 1 = Quite a bit', 9 =A great deal. 



Response 


Group 


Site 


A 


B 


C 


D 


E 


E 


All 


No answer 


Control 


0 


0 


0 


0 


0 


0 


0 


Experimental 


0 


0 


1 


0 


2 


0 


3 


Total 


0 


0 


1 


0 


2 


0 


3 


1 


Control 


1 


0 


1 


0 


0 


0 


2 


Experimental 


0 


0 


0 


0 


0 


0 


0 


Total 


1 


0 


1 


0 


0 


0 


2 


2 


Control 


0 


0 


1 


0 


0 


0 


1 


Experimental 


0 


0 


0 


0 


1 


0 


1 


Total 


0 


0 


1 


0 


1 


0 


2 


3 


Control 


7 


2 


6 


2 


1 


1 


19 


Experimental 


2 


5 


5 


0 


3 


1 


16 


Total 


9 


7 


11 


2 


4 


2 


35 


4 


Control 


1 


0 


1 


0 


1 


2 


5 


Experimental 


0 


0 


1 


1 


0 


1 


3 


Total 


1 


0 


2 


1 


1 


3 


8 


5 


Control 


0 


12 


6 


8 


9 


6 


41 


Experimental 


2 


10 


12 


10 


5 


4 


43 


Total 


2 


22 


18 


18 


14 


10 


84 


6 


Control 


0 


1 


0 


4 


2 


1 


8 


Experimental 


0 


0 


2 


3 


5 


3 


13 


Total 


0 


1 


2 


7 


7 


4 


21 


7 


Control 


3 


6 


4 


3 


5 


6 


27 


Experimental 


3 


7 


4 


7 


2 


5 


28 


Total 


6 


13 


8 


10 


7 


11 


55 


8 


Control 


0 


0 


0 


0 


0 


1 


1 


Experimental 


0 


0 


0 


0 


0 


1 


1 


Total 


0 


0 


0 


0 


0 


2 


2 


9 


Control 


0 


1 


0 


0 


1 


0 


2 


Experimental 


0 


0 


3 


0 


0 


2 


5 


Total 


0 


1 


3 


0 


1 


2 


7 




Total survey 
respondents 


19 


44 


47 


38 


37 


34 


219 
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Appendix Table J2. 

CTE-Teacher Perceptions of the Value of Math-Enhanced Lessons 



Site 


Group 


Total number of survey-takers 


CTE reinforces math skills with real-world applications; emphasis 
on math 


Math understanding is important to and/or strengthens CTE; 
essential to world of work; emphasis on CTE 


An overall benefit to students; good in general 


Demonstrates rationale/justification/relevance of the need for 


learning math; students accept math, whereas they had a knee- 
jerk emotional aversion to it before 


Helps CTE teacher learn more about math; lowers teacher’s 
anxieties 


Supports school initiatives or state standards in math 


Standardized exams 


Other response 


No answer 


A 


Control 


11 


1 


4 


4 


0 


0 


0 


1 


1 


2 




Experimental 


8 


1 


4 


1 


0 


0 


0 


0 


1 


1 




Both 


19 


2 


8 


5 


0 


0 


0 


1 


2 


3 


B 


Control 


22 


6 


9 


4 


3 


0 


0 


0 


0 


1 




Experimental 


22 


3 


10 


4 


4 


1 


0 


0 


2 


0 




Both 


44 


9 


19 


8 


7 


1 


0 


0 


2 


1 


C 


Control 


19 


3 


8 


8 


1 


1 


1 


0 


0 


2 




Experimental 


28 


5 


9 


8 


5 


0 


1 


0 


4 


1 




Both 


47 


8 


17 


16 


6 


1 


2 


0 


4 


3 


D 


Control 


17 


11 


1 


4 


4 


0 


0 


0 


2 


1 




Experimental 


21 


6 


2 


4 


6 


0 


1 


1 


0 


2 




Both 


38 


17 


3 


8 


10 


0 


1 


1 


2 


3 


E 


Control 


19 


11 


1 


4 


3 


0 


0 


0 


2 


1 




Experimental 


18 


10 


5 


8 


0 


1 


0 


0 


1 


0 




Both 


37 


21 


6 


12 


3 


1 


0 


0 


3 


1 


F 


Control 


17 


12 


1 


5 


1 


0 


0 


0 


0 


0 




Experimental 


17 


5 


5 


2 


2 


0 


0 


0 


0 


3 




Both 


34 


17 


6 


7 


3 


0 


0 


0 


0 


3 


All 


Control 


105 


44 


24 


29 


12 


1 


1 


1 


5 


7 




Experimental 


114 


30 


35 


27 


17 


2 


2 


1 


8 


7 




Both 


219 


74 


59 


56 


29 


3 


3 


2 


13 


14 
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Goals That CTE Teachers Expressed for their Partnerships with Math Teachers 
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APPENDIX K 

FINDINGS FROM POSTPROJECT CTE-TEACHER SURVEY PERTAINING TO 

PROFESSIONAL DEVELOPMENT 



Appendix Table Kl. 

Experimental-Group CTE-Teacher Responses Regarding the Extent to Which They Eelt 
Professional Development Prepared Them to Implement the Math-Enhancement Model 



Responses 


Sites 


A 


B 


C 


D 


E 


F 


All 


No. 


% 


No. 


% 


No. 


% 


No. 


% 


No. 


% 


No. 


% 


No. 


% 


Not at all 


0 


0.0 


1 


4.5 


0 


0.0 


1 


5.6 


0 


0.0 


0 


0.0 


2 


1.8 


To a small 
extent 


2 


28.6 


2 


9.1 


2 


7.7 


2 


11.1 


5 


27.8 


0 


0.0 


13 


11.9 


To some 
extent 


3 


42.9 


9 


40.9 


19 


73.1 


8 


44.4 


11 


61.1 


13 


72.2 


63 


57.8 


To a great 
extent 


2 


28.6 


10 


45.5 


5 


19.2 


7 


38.9 


2 


11.1 


5 


27.8 


31 


28.4% 



Typical Teacher Responses on Postproject CTE-Teacher Survey 
Regarding Improvement of Professional Development 

• need time to share and/or practice lessons and teaching strategies with others and/or 
improve by practice 

• need to improve existing lessons and support them with more materials/resources 

• need a better fit to curriculum (math in CTE) and/or improvements in lesson sequence 

• need math instruction for teachers and ideas for presenting math 

• professional development organization and structure 

• other responses 
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APPENDIX L 

VARIATION IN IMPLEMENTATION ACROSS REPLICATION SITES REGARDING 
MATH CONCEPTS ADDRESSED BY CTE MATH LESSONS 



Math Concept 


Number of Corresponding CTE Math Lessons 
Addressing the Math Concept 


E 


B 


E 


C 


D 


A 


Number & Number Relations 


1 


5 


10 


4 


5 


4 




compare, order 
















equivalent forms 




1 


3 


2 


1 






percentages 


1 


2 


10 


4 


3 


2 




exponents, scientific notation 




2 


1 




2 


4 




number line 




























Computation & Numerical Estimation 


7 


8 


9 


4 


4 


7 




computation 


6 


1 


7 




3 


2 




computation in context 


7 


8 


9 


4 


4 


7 
















Operation Concepts 


0 


2 


0 


0 


0 


0 




permutations, combinations 
















operation properties 




2 
























Measurement 


13 


7 


5 


4 


4 


2 




estimate 






1 


2 


1 


2 




rate 


2 


4 


5 


2 


1 


1 




scale drawing, map, model 


3 


1 






1 


1 




convert measurement units 


9 


4 




2 


2 






indirect measurement 
















mler use 


3 


1 




2 


2 


1 
















Geometry & Spatial Sense 


5 


0 


0 


1 


1 


0 




Pythagorean theorem 


2 














transformations 










1 






apply geometric properties 








1 








geometric constructions 


3 






1 




















Data Analysis, Statistics, & Probability 


4 


1 


22 


6 


4 


4 




interpret data display 


4 


1 


17 


5 


4 


3 




complete/construct data display 


3 


1 


18 


6 


4 


4 




make inferences from data 


1 




10 


5 


1 


3 




evaluate conclusions drawn from data 


2 




8 


1 


1 


3 




statistics 






11 


1 


2 


3 




probability 
















use data to solve problems 


3 




14 


5 


2 


4 




compare data 


1 




2 




1 


3 




describe, evaluate data 


1 




8 




1 


3 


1 















National Research Center for Career and Technical Education 



107 







Building Academic Skills in Context: Testing the Value of Enhanced Math Learning in CTE 



Patterns, Functions, & Algebra 


3 


5 


15 


4 


4 


5 




function 


3 




14 


2 


1 


5 




equation 


3 


2 


8 


2 


3 


3 




inequality 




2 




3 








graph quadratic equation 






2 










model problem situation 






3 


1 


1 






use algebra to solve problems 


1 


2 


6 




4 


3 
















Problem Solving & Reasoning 


3 


0 


1 


1 


0 


2 




develop, explain strategy 


3 














solve non-routine problem 








1 




2 




proportional reasoning 
















evaluate conjectures 






1 






















Communication 


1 


0 


0 


2 


1 


1 




model math situations 








2 


1 


1 




make conjectures 


1 






1 








evaluate ideas 
















explain thinking 








1 








explain solution process 


1 






1 




















Additional Math Concepts 


11 


3 


0 


3 


3 


0 




calculate perimeter/area/volume of a 
rectangle, circle, triangle 


7 


2 




2 


2 






calculate angles (trigonometry) 


4 


1 






1 






measure angles (compass and 
protractor) 


1 


1 




1 
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APPENDIX M 

MATH-IN-CTE LESSON PLAN TEMPLATE 



Lesson Title: 


Lesson No.: 


Author(s): 


Phone 

Number(s): 


E-Mail Address(es): 















Lesson 

Objective: 




Supplies 

Needed: 





THE SEVEN ELEMENTS 


TEACHER NOTES 
(and answer key) 


1. Introduce the CTE lesson. 




2. Assess students’ math awareness as it relates to 
the CTE lesson. 




3. Work through the math example embedded in 
the CTE lesson. 




4. Work through related, contextual math-in-CTE 
examples. 




5. Work through traditional math examples. 




6. Students demonstrate their understanding. 




7. Eormal assessment. 
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APPENDIX N 

LESSON PLAN RUBRIC 



Lesson Title: 



Lesson No.: 



Author(s): 



Please check the appropriate boxes in the rubric below. Use comment box to make suggestions/ 
recommendations. 



ELEMENTS 


COMPLETE 


NEEDS IMPROVEMENT 


COMMENTS 


1 . Introduce the 
CTE lesson. 


_ Specific objectives of 
CTE lesson are explicit. 

_ Detailed script is 

provided for introducing 
lesson to students as a 
CTE lesson. 

_ The pulled-out math 
concept embedded in the 
CTE lesson is clearly 
identified. 

_ Script is provided to 
point out the math in the 
CTE lesson. 


_ Lesson objectives are unclear 
or not evident. 

_ Little or no script is provided 
for introducing lesson to 
students. 

_ Math concept embedded in the 
CTE lesson is not pulled-out or 
made clear. 

_ Script is not provided to point 
out the math in the CTE lesson. 




2. Assess 

students’ math 
awareness as 
it relates to the 
CTE lesson. 


_ Lesson contains learning 
activities and/or well- 
developed questions that 
assess all students’ 
awareness of the 
embedded math concept. 

_ Math vocabulary and 
supporting instructional 
aids are provided to 
begin bridging of math to 
CTE. 


_ Script has short list of phrases; 
no learning activities or 
questions that support 
assessment of all students' 
awareness of the embedded 
math concept. 

_ Math vocabulary and/or 
instmctional aids are not 
provided. 




3. Work through 
the math 
example 
embedded in 
the CTE 
lesson. 


_ Script provides specific 
steps/processes for 
working through the 
embedded math example. 

_ CTE and math 
vocabularies are 
explicitly bridged in the 
script, supported with 
instmctional strategies 
and aids. 


_ Steps/processes for working 
through the embedded math 
example are incomplete or 
missing. 

_ Little bridging of CTE and 
math vocabularies is scripted; 
few or no strategies and aids 
are provided to relate the CTE 
to math. 
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4. Work through 
the related, 
contextual 
examples. 


_ Lesson provides a work- 
through of similar 
examples, using the same 
embedded math concept 
in examples from the 
same occupational area. 

_ Example problems are at 
varying levels of 
difficulty, from basic to 
advanced. 

_ Script continues to bridge 
the CTE and math 
vocabularies, supported 
with instructional 
strategies and/or aids. 


_ Few or no additional examples 
of the embedded concept are 
provided. 

_ Examples do not reflect 
varying levels of difficulty. 

_ Little or no bridging of CTE 
and math vocabularies is 
evident in the script, or 
supported with instructional 
strategies and/or aids. 




5. Work through 
traditional 
math 
examples. 


_ A variety of examples are 
scripted to illustrate the 
math concept as it is 
presented in traditional 
math tests. 

_ Examples move from 
basic to advanced. 

_ Script continues to bridge 
the CTE and math 
vocabularies, supported 
with instructional 
strategies and/or aids. 


_ Few or no math problems 

illustrate the math concept as it 
is presented in standardized 
tests. 

_ Examples do not reflect 
varying levels of difficulty. 

_ Little or no bridging of CTE 
and math vocabularies is 
evident in the script, or 
supported with instructional 
strategies and/or aids. 




6. Students 
demonstrate 
understanding. 


_ Lesson provides learning 
activities, projects, etc., 
that give students 
opportunities to 
demonstrate what they 
have learned. 

_ Lesson ties math 
examples back to the 
CTE content; lesson ends 
on the CTE topic. 


_ No learning activities, projects, 
etc., provide students with 
opportunities to demonstrate 
what they have learned. 

_ Lesson fails to tie the math 
back to CTE or end on the 
CTE topic. 




7. Formal 
assessment. 


_ Lesson provides 

questions/problems that 
will be included in 
formal assessments 
(tests, projects, etc.) in 
the CTE unit/course. 


_ Example questions/problems 
are not provided for use in 
formal assessments in the CTE 
unit/course. 
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APPENDIX O 

TEACHING REPORT FORMS 
Math Teacher Preteaching Report Form 

Submit as an e-mail attachment or by fax to your Site Director within 1 week following each 
preteaching review of a math-enhanced CTE lesson with your CTE instructor. 



Your ID #: 


CTE teacher’s ID #: 


Date of review: 


Title of lesson reviewed: 


Lesson #: 



Answer the following questions by putting an X in the box on the scale following each question 
that best reflects your opinion: 



1 . 

2 . 

3. 

4. 



In your judgment, how well are the math concepts 
integrated into the occupational content of this lesson? 

How adequate is the amount/depth of instruction in this 
lesson to teach students the math concepts? 

How would you rate the CTE instructor’s “comfort” with 
teaching the math in this lesson? 

How much assistance do you think you gave the CTE 
instructor? 



Not at all Completely 



1 1 

Not 


1 1 

at all 


1 1 


1 1 


1 1 

Cc 


1 1 

imple 


1 1 

fely 


Lov 


/ 








I 


ligh 


Nor 


le 








P 


i lot 



5. Are all seven elements of the math-enhancement model clearly presented in the lesson? 
Yes No If no, what elements are weak or missing? 



6. What part(s) of the math in this lesson did the CTE instructor need the most assistance with? 



7. Do you have any suggestions for improving the teaching of the math concepts in this lesson? 



National Research Center for Career and Technical Education 



113 







Building Academic Skills in Context: Testing the Value of Enhanced Math Learning in CTE 



CTE Teacher Postteaching Report Form 

Submit as an e-mail attachment or by fax to your Site Director within 1 week following each 
teaching of a math-enhanced CTE lesson. 



Your ID #: 


Math teacher’s ID #: 


Date(s) lesson taught: 


Title of lesson taught: 


Lesson #: 


Total class time, in minutes, spent on this lesson: 


Total number of classes in which the 
lesson was taught: 



Answer the following questions by putting an X in the box on the scale following each question 
that best reflects your opinion: 



1 . 

2 . 

3. 

4. 

5. 

6 . 



7. 



In your judgment, how well were the math concepts 
integrated into the occupational content of this lesson? 

How adequate is the amount/depth of instruction in this 
lesson to teach students the math concepts? 

How would you rate your “comfort” with teaching the 
math in this lesson? 

How much assistance did you receive from your math 
partner prior to teaching this lesson? 

To what degree do you think your students learned the 
math in this lesson? 

Overall, how successful was the lesson, in both CTE 
and math components? 



Not at all 



Completely 



1 L 

Not 


1 

at all 






1 

C( 


1 

)mpk 


1 1 

;tely 


Low 










I 


ligh 


Non 


e 








t 


^ lot 


A lit 


tie 










^ lot 


Not 


at all 






C( 


)mpk 


;tely 



Were you able to complete the lesson as planned? Yes No_ 



a. If no, what prevented you from completing it? 



8. Were you able to teach all seven elements of the math-enhancement model? 

Yes No If no, what elements were not included? 

9. Do you have any suggestions for improving the teaching of the CTE content or the math 
concepts in this lesson? 
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